Distilling Knowledge for Fast Retrieval-based Chat-bots
Table 2. Prediction quality metrics across all datasets. Metrics for models trained with knowledge distillation, which are significant relative to models trained without it, are marked in bold. We use paired two-tailed t-tests with a p-value¡0.05 to perform significance tests. For easier reading metrics have been multiplied by 100. No data augmentation has been used and training samples are used as is. +KD indicates a model trained with knowledge distillation.
['(lr)2-3(lr)4-5(lr)6-7(lr)8-9', 'UDC20% R@1', 'UDC20% MRR', 'UDC R@1', 'UDC MRR', 'MANtIS R@1', 'MANtIS MRR', 'DSTC7 R@1', 'DSTC7 MRR']
These results indicate that employing a task-specific single layer cross-attention mechanism on top of separately encoded inputs is highly effective for the task of response retrieval. Of particular note is the increased gap between the performance of the two methods when using smaller training sets (UDC20%, MANtIS, DSTC7). This shows that the regular bert-cross model struggles when fine-tuned with smaller response-retrieval sets and data augmentation or a some other method must be used to achieve acceptable results. In contrast, our enhanced BERT cross-encoder’s R@1 only dropped by 3.3 points when its training set was reduced to a fifth.