Teaching Machines to Converse
Table 6.3: The gain from the proposed adversarial model over the mutual information system based on pairwise human judgments.
['Setting', 'adver-win', 'adver-lose', 'tie']
, we follow protocols defined in Chapter 5, employing crowdsourced judges to evaluate a random sample of 200 items. We present both an input message and the generated outputs to 3 judges and ask them to decide which of the two outputs is better (single-turn general quality). They were instructed to prefer outputs that were more specific (relevant) to the message and preceding context, Ties are permitted. Identical strings are assigned the same score. We also present the judges with multi-turn conversations simulated between the two agents. Each conversation consists of 3 turns. We observe a significant quality improvement on both single-turn quality and multi-turn quality from the proposed adversarial model. It is worth noting that the reinforcement learning system described in Chapter 5, which simulates conversations between two bots and is trained based on manually designed reward functions, only improves multi-turn dialogue quality, while the model described in this chapter improves both single-turn and multi-turn dialogue generation quality. This confirms that the reward adopted in adversarial training is more general, natural, and effective in training dialogue systems.