Improving Matching Models with Hierarchical Contextualized Representations for Multi-turn Response Selection
TABLE II: Evaluation results of model ablation.
['[EMPTY]', '[BOLD] Ubuntu Corpus', '[BOLD] Ubuntu Corpus', '[BOLD] Ubuntu Corpus', '[BOLD] Ubuntu Corpus', '[BOLD] Douban Corpus', '[BOLD] Douban Corpus', '[BOLD] Douban Corpus', '[BOLD] Douban Corpus', '[BOLD] Douban Corpus', '[BOLD] Douban Corpus']
First, we find that both ECMoL and ECMoG are useful to matching. ECMoG generally brings more improvement to matching than ECMoL for Multi-view while the results reverse for SMN. This phenomenon is mainly due to the different interaction operation of the two models. In Multi-view, the response and context are interacted with corresponding global utterance representations, while in SMN, the interaction is performed word-by-word. Second, fine-tuning is necessary as the performance of models drops without it. Since the Ubuntu dialogues have some domain-specific knowledge, a HED model only trained on Twitter cannot fully capture the contextual information in the Ubuntu data, resulting in inaccurate contextualized word representations. We can see that ECMoG and ECMoL shows different robustness to the noise for Multi-view and SMN due to the different interaction operation. Finally, although fine-tuning is important, one cannot only train a HED model on the domain-specific data, because the size of the data is small and a lot of contextual information in common conversation language will be lost. That explains why model+ECMo (Ubuntu-only) is inferior to all other variants.