Non-Autoregressive Machine Translation with Latent Alignments
Table 5: WMT’14 En-De BLEU comparison for distillation base vs big Transformer, and number of decoding iterations.
['[BOLD] Model', 'Iterations', '[BOLD] En→De', '[BOLD] De→En']
Our Imputer model using 8 decoding iterations achieves 28.2 BLEU on En→De, slightly outperforming the autoregressive Transformer of 27.8 BLEU. On De→En, we achieve 31.3 BLEU, on par with the autoregressive Transformer model. Similarly, on En↔Ro, Imputer matches the performance of the autoregressive teacher using just 4 decoding iterations. We also observe robustness of our Imputer model when we reduce the number of decoding iterations from 8 to 2. Using just 2 iterations, we obtain 27.5 BLEU on En→De and 30.2 BLEU on De→En. These results were trained with distillation from a big Transformer model.