DailyDialog: A Manually Labelled Multi-turn Dialogue Dataset
Table 4: Experiments Results of generation-based approaches.
['[EMPTY]', 'Epoch', 'Test Loss', 'PPL', 'BLEU-1', 'BLEU-2', 'BLEU-3', 'BLEU-4']
Intention and Emotion-enhanced To utilize the intention and emotion labels, we follow Zhou et al. The intention and emotion labels are characterized as one-hot vectors. (last four columns), we can see that in general attention-based approaches are better than vanilla Seq2Seq model. Among the three compared approaches, HREDs achieve highest BLEU scores because they take history information into consideration. Furthermore, label information is effective even though we utilize them in the simplest way. These findings are consistent with previous work Sordoni et al. ; Serban et al. OpenSubtitle converge faster, achieving lower Perplexity (PPL) but poorer BLEU scores. We conjecture it as a result of domain difference. OpenSubtitle dataset is constructed by movie lines, whereas our datasets are daily dialogues. Moreover, OpenSubtitles has approximately 1000+ speaker turns in one “conversation”, while our dataset has in average 8 turns. To pretrain a model by corpus from different domain will harm its performance on the target domain. Hence, it is less optimal to simply pretrain models with large-scale datasets such as OpenSubtitle, which is domain different from the evaluation datasets. We further examine this issue by comparing the generated answers by models trained solely on DailyDialog with and without pre-training.