Towards Generating Long and Coherent Text with Multi-Level Latent Variable Models
Table 3: Evaluation results for generated sequences by our models and baselines on corpus-level BLEU scores (B-n denotes the corpus-level BLEU-n score.)
['[BOLD] Model', '[BOLD] Yelp  [BOLD] B-2', '[BOLD] Yelp  [BOLD] B-3', '[BOLD] Yelp  [BOLD] B-4', '[BOLD] arXiv  [BOLD] B-2', '[BOLD] arXiv  [BOLD] B-3', '[BOLD] arXiv  [BOLD] B-4']
VAE tends to be a stronger baseline for paragraph generation, exhibiting higher corpus-level BLEU scores than both AAE and ARAE. The VAE with multi-level decoder demonstrates better BLEU scores than the one with a flat decoder, indicating that the plan-ahead mechanism associated with the hierarchical decoding process indeed benefits the sampling quality. ml-VAE-D exhibits slightly better results than ml-VAE-S. We attribute this to the more flexible prior distribution of ml-VAE-D, which improves the ability of inference networks to extract semantic features from a paragraph, yielding more informative latent codes.