Domain Adaptive Training BERT for Response Selection
Table 3: Variable fine-tuning in BERT. All results are evaluated using mean reciprocal rank (MRR).
['[BOLD] Variable Fine-Tuning', '[BOLD] Ubuntu', '[BOLD] Advising']
Houlsby et al. They suggested that small data sets may be sub-optimal, when fine-tuning the whole BERT layers. Inspired by this assumption, we conduct an experiment The models achieve the best performance when T=4 and T=6, for Ubuntu and Advising, respectively. We demonstrate that utilizing this application is effective on not only small sets but also domain-specific sets. We also experiment how effective data augmentation is for both data sets. Formerly, both training sets contain 1:1 ratio of positive and negative responses. We change the ratio of the samples to 1:4 by increasing negative samples. The process of how we select negative samples is heuristic, and the best performance is obtained at ration of 1:4. It actually affects the performance improvement of Ubuntu, but not for Advising. We hypothesize that the Advising Corpus, unlike Ubuntu, is created with a number of sub-dialogs extracted from the original one. We obtain 0.3% higher R10@1 compared with the model trained on original one. We set same experiment conditions with the BERT-VFT that shows the best performance.