Grounded Conversation Generation as Guided Traverses in Commonsense Knowledge Graphs
Table 7: More Metrics on Relevance of Generated Responses. The relevance is calculated between the generated response and the golden response. Concept-PPL is the method used for calculating Perplexity in CCM (Zhou et al., 2018a), which combines the distribution of both words and concepts together. The Concept-PPL is meaningless when utilizing different numbers of concepts (more concepts included, better Perplexity shows).
['Model', 'Bleu-1', 'Bleu-2', 'Bleu-3', 'Nist-1', 'Nist-2', 'Nist-3', 'Concept-PPL']
Commonsense knowledge aware conversation generation with graph attention.. In Proceedings of IJCAI, pp. 4623–4629. External Links: ConceptFlow outperforms other baselines with large margins among all evaluation metrics. Concept-PPL is the Perplexity that calculated by the code from previous work Zhou et al. Zhou et al. It is evident that more entities will lead to a better result in terms of Concept-PPL because the vocabulary size of entities is always smaller than word vocabulary size. These supplementary results compare the generated response with the user post to measure the repeatability of the post and generated responses. A lower score indicates better performance because the repetitive and dull response will degenerate the model performance. ConceptFlow presents competitive performance with other baselines, which illustrate our model provides an informative response for users.