Towards Generating Long and Coherent Text with Multi-Level Latent Variable Models
Table 2: Language modeling results on Yelp and arXiv data. Upper block are baselines, and lower are our models.
['[BOLD] Model', '[BOLD] Yelp  [BOLD] NLL', '[BOLD] Yelp  [BOLD] KL', '[BOLD] Yelp  [BOLD] PPL', '[BOLD] arXiv  [BOLD] NLL', '[BOLD] arXiv  [BOLD] KL', '[BOLD] arXiv  [BOLD] PPL']
The flat-VAE model obtains slightly worse NNL and PPL relative to a flat LSTM-based language model. With multi-level LSTM decoder, our ml-VAE-S yields increased KL divergence, demonstrating that the VAE model tends to leverage more information from the latent variable in the decoding stage. The PPL of ml-VAE-S is also decreased from 47.9 to 46.6 (compared to ml-LM), indicating that the sampled latent codes improve word-level predictions.