Learning Discourse-level Diversity for Neural Dialog Models using Conditional Variational Autoencoders
Table 3: The reconstruction perplexity and KL terms on Penn Treebank test set.
['Model', 'Perplexity', 'KL cost']
The standard VAE fails to learn a meaningful latent variable by having a KL cost close to 0 and a reconstruction perplexity similar to a small LSTM LM Zaremba et al. KLA helps to improve the reconstruction loss, but it requires early stopping since the models will fall back to the standard VAE after the KL weight becomes 1. At last, the models with BOW loss achieved significantly lower perplexity and larger KL cost.