Pre-Trained and Attention-Based Neural Networks for Building Noetic Task-Oriented Dialogue Systems
Table 4: Ablation results for a single model on the validation set of Ubuntu dataset in subtask 1.
['[EMPTY]', 'Recall@1', 'Recall@5', 'Recall@10', 'MRR']
Without the pre-training process, the metric of Recall@1 drops a large margin 1.6%, which shows that the external data given by the DSTC 8 package does improve our model further when we use pre-training. Furthermore, the performance continues to drop 0.6% in terms of Recall@1 by ablating the switch embeddings, which shows the effectiveness of utilizing the information of speaker turns.