Towards Generating Long and Coherent Text with Multi-Level Latent Variable Models
Table 7: Human evaluations on Yelp Reviews dataset. Each block is a head-to-head comparison of two models on grammatically, consistency, and non-redundancy.
['[BOLD] Model', '[BOLD] Grammar.', '[BOLD] Cons.', '[BOLD] Non-Red.', '[BOLD] Overall']
As shown in Even though both models underperform when compared against the ground-truth real reviews, ml-VAE was rated higher in comparison to flat-VAE (raters find ml-VAE closer to human-generated than the flat-VAE) in all the criteria evaluation criteria. When compared against AAE baseline models using the same data preprocessing steps and hyperparameters, ml-VAE again produces more grammatically-correct and semantically-coherent samples. The human evaluations correlate with the automatic metrics, which indicate that our ml-VAE is actually generating more coherent stories than the baseline models. We leave further evaluations using embedding based metrics as a possible extension to our work.