Sequential Sentence Matching Network for Multi-turn Response Selection in Retrieval-based Chatbots
Table 2: Experiment results of our model and other baseline methods on three benchmark datasets. The subscript pure represents the model only containing sentence matching; I1, I2 and I3 represents the model integrates with IOI in representation, matching and aggregation stages respectively.
['Models', 'Ubuntu Corpus  [BOLD] R10@1', 'Ubuntu Corpus  [BOLD] R10@2', 'Ubuntu Corpus  [BOLD] R10@5', 'Douban Corpus  [BOLD] MAP', 'Douban Corpus  [BOLD] MRR', 'Douban Corpus  [BOLD] P@1', 'Douban Corpus  [BOLD] R10@1', 'Douban Corpus  [BOLD] R10@2', 'Douban Corpus  [BOLD] R10@5', 'Ecommerce Corpus  [BOLD] R10@1', 'Ecommerce Corpus  [BOLD] R10@2', 'Ecommerce Corpus  [BOLD] R10@5']
All the results except ours are directly from the corresponding papers. From the table, we can see that S2Mpure achieves a competitive result on three datasets. This result is a strong evidence showing that sequential sentence matching is another effective method in multi-turn response selection task. Moreover, S2MI3 is consistently better than the current best performing method by a margin of 1.7% in terms of R10@1 on Ubuntu Dialogue Corpus; 0.9% in terms of MRR and 1.4% in terms of R10@1 on Douban Conversation Corpus; and 5.6% in terms of R10@1 on E-commerce Dialogue Corpus, achieving a new state-of-the-art performance on three datasets. It should be noticed that the negative candidates in E-commerce Dialogue Corpus are retrieved according to some key words in the context, which strongly confuses the models matching in word-level and segment-level with similarity matrices. Such a great margin of improvement can strongly support our standpoint that sentence matching mechanism can address the problem that we discussed in the introduction.