Do sequence-to-sequence VAEs learn global features of sentences?
Table 8: Resetting the decoder brings very noticeable gains on all data-regimes and with different rates. Yelp dataset, δ-VAE free bits, no KL annealing. For columns interpretations, see Table 7.
['Reset.', '[ITALIC] λ', 'F1(5)', 'F1(50)', 'F1(500)', 'F1(5000)', 'F1(all)', 'KL(5)']
Li et al. Then, the parameters of the decoder are re-initialised and the (modified) KL term is added to the objective. Since it is not very clear why it would be useful, we studied the impact of this choice.