\mathbf{\mu}-Forcing: Training Variational Recurrent Autoencoders for Text Generation
Table 2. The evaluation results of different settings on the APRC test set.
['Setting', 'Rec', 'KL', 'BLEU-4', 'BLEU-5', 'SLEU-4', 'SLEU-5']
We report the reconstruction loss, KL loss on the test set. From the results, we can see that the vanilla hybrid model suffers from the uninformative latent variables issue, result in high reconstruction error and KL vanishing on the test set. However, both auxiliary reconstruction term and the proposed μ-Forcing term Lμ can solve this issue. Furthermore, when applied our method to the vanilla hybrid model, it achieves lower reconstruction loss, SLEU scores and higher BLEU scores compare with their full model (Hybrid + Aux). These results indicate that the proposed method can be also applied to HybridVAE and further improve the performance.