Generating Sentences from Disentangled Syntactic and Semantic Spaces
Table 1: BLEU and Forward PPL of VAE with varying KL weights on the PTB test set. The larger↑ (or lower↓), the better.
['KL-Weight', 'BLEU↑', 'Forward PPL↓']
S4SS1SSS0Px4 Results that BLEU and PPL are more or less contradictory. Usually, a smaller KL weight makes the autoencoder less “variational” but more “deterministic,” leading to less fluent sampled sentences but better reconstruction. If the trade-off is not analyzed explicitly, the VAE variant could have arbitrary results based on KL-weight tuning, which is unfair.