An EM Approach to Non-autoregressive Conditional Sequence Generation
Table 6: Analysis of the amortized inference for iteration t=2,3,4 on WMT En-De test BLEU. All the models are evaluated without ODD decoding and rescoring. We show the results of the NAR models using different training data in the M-step.
['[EMPTY]', 'amortized', 'non-amortized']
In our EM method, we employ amortized inference and parametrize the posterior distribution of the target sequences by an AR model. In this section, we investigate the importance of amortized inference. Specifically, we try to directly train the NAR model on (Xt+1,Yt+1) in the M-step. The results are shown in Tab. We can see that parameterizing the posterior distribution by a unified AR model always improves the performance of the NAR model.