Adversarial Mutual Information for Text Generation
Table 1: Quantitative evaluation for dialog generation on the PersonaChat dataset. “LNS” denotes the latent noise sampling. The top part presents the baselines, the medium and bottom parts show the performance and ablation results of our AMI framework based on the LSTM and Transformer. Our AMI significantly improves both LSTM and Transformer and achieves the state-of-the-art results.
['[BOLD] Models', '[BOLD] Relevance  [BOLD] Average', '[BOLD] Relevance  [BOLD] Greedy', '[BOLD] Relevance  [BOLD] Extrema', '[BOLD] Diversity  [BOLD] \xa0Dist-1', '[BOLD] Diversity  [BOLD] \xa0Dist-2', '[BOLD] Diversity  [BOLD] \xa0Ent-4']
As we can see, our method with Transformer clearly outperforms all the baselines and achieves state-of-the-art results. From the medium and bottom parts, we observe that our AMI greatly improves the relevance and diversity performances over both the LSTM and Transformer. Compared to MMI, our method significantly exceeds on both the relevance and diversity metrics, which indicates that our adversarial training scheme helps improve the quality of the backward model, which subsequently provides more credible rewards to the forward network. We also conduct ablation experiments that when we use beam search for sampling instead of our proposed latent sampling noise, we find that the results drop on both metrics, which demonstrates that this sampling strategy can enlarge the searching space from the high-level structure while maximizing the mutual information between the output and the source text.