Adversarial Mutual Information for Text Generation
Table 2: Embedding average for the output of the backward network, where the input is the utterance (1) generated by the forward network or (2) from the dataset created by human.
['[BOLD] Backward Network', '[BOLD] Synthetic', '[BOLD] Real']
We evaluate the embedding average of the outputs of the backward network when feeding it with the synthetic data and the real data. Compared to the MMI with fixed ϕ (i.e. the pre-trained model), the performance of the regular MMI backward model drops and its output text is more relevant to ground truth when fed with the synthetic data rather than the real data. This means such a training framework misleads the backward model to give lower rewards to the generated targets that look more like the real ones, and thus results in negative effect to the optimization of the forward model. With our adversarial training scheme, the backward model performs better when fed with the real data and worse with the synthetic data, which means it improves the ability to manage the information flow.