You Impress Me: Dialogue Generation via Mutual Persona Perception
Table 1: Automatic evaluation results of different methods on the Persona-Chat dataset. The standard deviation [σ] (across 5 runs) of P2 \textscBot is also reported. All the results were evaluated on the dev set since the test set was not publicly available.
['[BOLD] Category', '[BOLD] Model', '[BOLD] Original Hits@1(%)\u2009↑', '[BOLD] Original ppl\u2009↓', '[BOLD] Original F1(%)\u2009↑', '[BOLD] Revised Hits@1(%)\u2009↑', '[BOLD] Revised ppl\u2009↓', '[BOLD] Revised F1(%)\u2009↑']
Following Zhang et al. Given 20 response candidates, Hits@1 is the probability that the real response ranks the highest according to the model. Perplexity measures the negative log likelihood of the correct sequence output by the model, lower values indicating better performance. F1 is the harmonic mean of word-level precision and recall. As observed, our approach outperforms almost all baselines and achieves new state-of-the-art performance on ppl and F1, with highly competitive performance on Hits@1. In the revised mode, our approach still achieves the best performance, obtaining a relative improvement of 13.4% on F1 against the strongest baseline. It is worth noting that we also tried to employ F1 as the reward, but the result is far from satisfactory.