Sequential Matching Network: A New Architecture for Multi-turn Response Selection in Retrieval-Based Chatbots
Table 4: Evaluation results of model ablation.
['[EMPTY]', '[BOLD] Ubuntu Corpus', '[BOLD] Ubuntu Corpus', '[BOLD] Ubuntu Corpus', '[BOLD] Ubuntu Corpus', '[BOLD] Douban Conversation Corpus', '[BOLD] Douban Conversation Corpus', '[BOLD] Douban Conversation Corpus', '[BOLD] Douban Conversation Corpus', '[BOLD] Douban Conversation Corpus', '[BOLD] Douban Conversation Corpus']
First, replacing the multi-channel “2D” matching with a neural tensor network (NTN) Socher et al. (denoted as ReplaceM) makes the performance drop dramatically. This is because NTN only matches a pair by an utterance vector and a response vector and loses important information in the pair. Together with the visualization, we can conclude that “2D” matching plays a key role in the “matching first” strategy as it captures the important matching information in each pair with minimal loss. Second, the performance drops slightly when replacing the GRU for matching accumulation with a multi-layer perceptron (denoted as ReplaceA). This indicates that utterance relationships are useful. Finally, we left only one channel in matching and found that M2 is a little more powerful than M1 and we achieve the best results with both of them (except on R10@5 on the Douban Corpus).