ReCoSa: Detecting the Relevant Contexts with Self-Attention for Multi-turn Dialogue Generation
Table 2: The metric-based evaluation results (%).
['JDC Dataset model', 'JDC Dataset PPL', 'JDC Dataset BLEU', 'JDC Dataset distinct-1', 'JDC Dataset distinct-2']
From the results, we can see that the attention-based models, such as WSeq, HRAN and HVMN, outperform the traditional HRED baselines in terms of BLEU and distinct-2 measures. That’s because all these models further consider the relevance of the contexts in the optimization process. HRAN uses a traditional attention mechanism to learn the importance of the context sentences. HVMN uses a memory network to remember the relevant context. But their effects are both quite limited. Our proposed ReCoSa performs the best. Take the BLEU score on JDC dataset for example, the BLEU score of ReCoSa model is 13.797, which is significantly better than that of HRAN and HVMN, i.e., 12.278 and 13.125. The distinct scores of our model are also higher than baseline models, which indicate that our model can generate more diverse responses. We have conducted the significant test, and the result shows that the improvements of our model are significant on both Chinese and English datasets, i.e., p-value<0.01. In summary, our ReCoSa model has the ability to produce high quality and diverse responses, as compared with baseline methods.