Improved Variational Neural Machine Translation by Promoting Mutual Information
Table 1: Results on improving posterior collapse. The KL value refers to DKL(qϕ(z∣x,y)∥p(z∣x)) for DCVAE and DKL(qϕ(z∣y)∥p(z∣x)) for our model.
['Model', 'DKL', 'I [ITALIC] qϕ( [ITALIC] z, [ITALIC] x)', 'I [ITALIC] qϕ( [ITALIC] z, [ITALIC] y)', 'NLL']
The posterior collapse of the baseline model is apparent: both DKL mutual information terms drop to 0 at the beginning of training as a result ELBO’s design. On the other hand, our model, without using any annealing schedule, can effectively increase mutual information and prevent KL loss from settling to a degenerate solution early on. disable ,author=R3todo: disable,author= R3The comparison of KL term in table 1 seems meaningless, because the term in different methods are different. disable ,author=Aryatodo: disable,author= disable,author=Xiantodo: disable