Sequential Attention-based Network for Noetic End-to-End Response Selection
Table 4: Ablation analysis on the development set for the DSTC7 Ubuntu dataset.
['[BOLD] Sub', '[BOLD] Models', '[BOLD] R@1', '[BOLD] R@10', '[BOLD] R@50', '[BOLD] MRR']
For Ubuntu subtask 1, ESIM achieved 0.854 R@10 and 0.6401 MRR. If we removed context’s local matching and matching composition to accelerate the training process (“-CtxDec”), R@10 and MRR dropped to 0.845 and 0.6210. Further discarding the last words instead of the preceding words for the context (“-CtxDec & -Rev”) degraded R@10 and MRR to 0.840 and 0.6174. Ensembling the above three models (“Ensemble”) achieved 0.887 R@10 and 0.6790 MRR. Ensembling was performed by averaging output from models trained with different parameter initializations and different structures.