Improving Matching Models with Hierarchical Contextualized Representations for Multi-turn Response Selection
TABLE III: The results of the models with fixed or continue-trained ECMo representations.
['[EMPTY]', '[BOLD] Ubuntu Corpus', '[BOLD] Ubuntu Corpus', '[BOLD] Ubuntu Corpus', '[BOLD] Ubuntu Corpus', '[BOLD] Douban Corpus', '[BOLD] Douban Corpus', '[BOLD] Douban Corpus', '[BOLD] Douban Corpus', '[BOLD] Douban Corpus', '[BOLD] Douban Corpus']
Does further optimization under the matching loss help? So far, we fix the pre-trained ECMo representations in matching. We find that “continue-train” makes the performance of Multi-view and SMN drop on both datasets. This phenomenon may stem from the fact that the matching model is prone to over-fitting with the additional parameters coming from HED. On the other hand, fine-tuning with cross-entropy loss may damnify the original contextualized representations.