An Auto-Encoder Matching Model for Learning Utterance-Level Semantic Dependency in Dialogue Generation
Table 3: Human evaluation results of the AEM model and the Seq2Seq model.
['Models', 'Fluency', 'Coherence', 'G-Score']
The inter-annotator agreement is satisfactory considering the difficulty of human evaluation. The Pearson’s correlation coefficient is 0.69 on coherence and 0.57 on fluency, with p<0.0001. First, it is clear that the AEM model outperforms the Seq2Seq model with a large margin, which proves the effectiveness of the AEM model on generating high quality responses. Second, it is interesting to note that with the attention mechanism, the coherence is decreased slightly in the Seq2Seq model but increased significantly in the AEM model. It suggests that the utterance-level dependency greatly benefits the learning of word-level dependency. Therefore, it is expected that the AEM+Attention model achieves the best G-score.