You Impress Me: Dialogue Generation via Mutual Persona Perception
Table 2: Human evaluation results.
['Model', '1\u2009(%)', '2\u2009(%)', '3\u2009(%)', '4\u2009(%)', 'Avg']
As mentioned in Dinan et al. Hence, we also performed crowd-sourced human evaluations on the state-of-the-art baselines (i.e. Transfertransfo & Lost In Conversation) and our proposed P2 \textscBot. Concretely, on the original dev set, we randomly sampled 200 responses generated by these methods and asked each worker to rate them. The rating ranges from 1 to 4. 1 means the response is good only in terms of grammar and sentence structure; 2 means in addition to valid grammar, the response is also coherent with the context; 3 means the coherent response is meanwhile interesting and informative, instead of just a simple response like “Yes”; And 4 means the response is consistent with the persona of the interlocutor, which is of extreme importance for the task of reflecting whether the model can effectively utilize the persona information. We also conducted Wilcoxon signed-rank tests between our method and the baselines and the results show the improvements are significant with p < 0.05.