ReCoSa: Detecting the Relevant Contexts with Self-Attention for Multi-turn Dialogue Generation
Table 4: The human evaluation on JDC and Ubuntu.
['JDC Dataset model', 'JDC Dataset ReCoSa vs.', 'JDC Dataset ReCoSa vs.', 'JDC Dataset ReCoSa vs.', 'JDC Dataset kappa']
4.2.2 Human Evaluation The percentage of win, loss and tie, as compared with the baselines, are given to evaluate the quality of generated responses by ReCoSa. From the results, we can see that the percentage of win is always larger than that of loss, which shows that our ReCoSa model significantly outperforms baselines. Take JDC as an example. Compared with HRAN, WSeq and HVMN, the ReCoSa achieves preference gains (win subtracts loss) 10.35%, 25.86% and 13.8%, respectively. We also conducted the significant test, and the result shows that the improvements of our model are significant on both two datasets, i.e., p-value<0.01.