Personalizing Dialogue Agents: I have a dog, do you have pets too?
Table 5: Evaluation of dialog utterance prediction with generative models in four settings: conditioned on the speakers persona (“self persona”), the dialogue partner’s persona (“their persona”), both or none. The personas are either the original source given to Turkers to condition the dialogue, or the revised personas that do not have word overlap. In the “no persona” setting, the models are equivalent, so we only report once.
['[BOLD] Persona', '[BOLD] Method', '[BOLD] Original  [BOLD] ppl', '[BOLD] Original  [BOLD] hits@1', '[BOLD] Original  [BOLD] F1', '[BOLD] Revised  [BOLD] ppl', '[BOLD] Revised  [BOLD] hits@1', '[BOLD] Revised  [BOLD] F1']
Their Persona. Using “Their persona” has less impact on this dataset. We believe this is because most speakers tend to focus on themselves when it comes to their interests. It would be interesting how often this is the case in other datasets. Certainly this is skewed by the particular instructions one could give to the crowdworkers. For example if we gave the instructions “try not to talk about yourself, but about the other’s interests’ likely these metrics would change. In particular we give results for conditioning on the other person’s persona (”Their”) instead of one’s own (”Self”) or the combination (”Both”). In particular, we see that, knowing the other’s persona does not help precision on this data with these models. Revised personas give improved results, perhaps because the models are forced to learn more than trivial word overlap (i.e. semantic similarity of differing phrases).