A Surprisingly Effective Fix for Deep Latent Variable Modeling of Text
Table 3: Language modeling results on PTB and Yahoo test set. We bold the lines that represent the best average of language modeling and reconstruction loss. Cyclic⋆ is from (Liu et al., 2019).
['[BOLD] Method', '[BOLD] PPL↓', '[BOLD] Recon↓', '[BOLD] AU↑', '[BOLD] KL', '[BOLD] -ELBO']
Meanwhile, for representation learning, our method beats all the baselines by a large margin on reconstruction loss with all latent units active. We also note that our method is not very sensitive to the target rate λ.