Non-autoregressive Machine Translation with Disentangled Context Transformer
Table 1: The performance of non-autoregressive machine translation methods on the WMT14 EN-DE and WMT16 EN-RO test data. The Step columns indicate the average number of sequential transformer passes. Shaded results use a small transformer (d_{model}=d_{hidden}=512). Our EN-DE results show the scores after conventional compound splitting (luong-etal-2015-effective; Vaswani2017AttentionIA).
['[BOLD] Model n: # rescored candidates', '[BOLD] en\\rightarrowde Step', '[BOLD] en\\rightarrowde BLEU', '[BOLD] de\\rightarrowen Step', '[BOLD] de\\rightarrowen BLEU', '[BOLD] en\\rightarrowro Step', '[BOLD] en\\rightarrowro BLEU', '[BOLD] ro\\rightarrowen Step', '[BOLD] ro\\rightarrowen BLEU']
First, our re-implementations of CMLM + Mask-Predict outperform Ghazvininejad2019MaskPredictPD (e.g., 31.24 vs. 30.53 in de\rightarrowen with 10 steps). This is probably due to our tuning on the dropout rate and weight averaging of the 5 best epochs based on the validation BLEU performance (Sec.