Beyond Domain APIs: Task-oriented Conversational Modeling with Unstructured Knowledge Access
Table 5: Automated evaluation results on knowledge-grounded response generation
['Method', 'PPL', 'Unigram F1', 'Div. ( [ITALIC] n = 1)', 'Div. ( [ITALIC] n = 2)', 'BLEU-4', 'METEOR', 'ROUGE-L']
Responses by answer extraction The evaluation was done only on the augmented turns with the ground-truth knowledge, in order to characterize the models’ ability to handle the external knowledge scenario. Especially, the GPT-2 model with knowledge outperformed both the answer extraction baseline and the other GPT-2 variant with no knowledge in BLEU-4, METEOR, and ROUGE-L, which indicates that our proposed neural model generates more human-like responses than the extractive baseline.