Improving Matching Models with Hierarchical Contextualized Representations for Multi-turn Response Selection
TABLE I: Evaluation results on the two datasets. Numbers in bold mean that improvement to the original models brought by ECMo is statistically significant (t-test, p-value <0.01 ). Numbers marked with ∗ mean that improvement to ELMo (fine-tune) and CoVe is statistically significant (t-test, p-value <0.01 ). We do not include the results of CoVe and ELMo enhanced models on the Douban data because the two models are not available for Chinese data.
['[EMPTY]', '[BOLD] Ubuntu Corpus', '[BOLD] Ubuntu Corpus', '[BOLD] Ubuntu Corpus', '[BOLD] Ubuntu Corpus', '[BOLD] Douban Corpus', '[BOLD] Douban Corpus', '[BOLD] Douban Corpus', '[BOLD] Douban Corpus', '[BOLD] Douban Corpus', '[BOLD] Douban Corpus']
We can observe that the performance of Multi-view and SMN improves on almost all metrics after they are combined with CoVe, ELMo (fine-tune), and ECMo, indicating that contextualized vectors are useful to the multi-turn response selection task. Notably, ECMo brings more significant and more consistent improvement to matching models on both datasets, which verifies the effectiveness of the proposed method. With ECMo, a simple Multi-view even performs better than SMN on the Ubuntu data, although SMN is in a more complicated structure, and thus is capable of capturing more semantic information in conversational contexts.