ENGINE: Energy-Based Inference Networks for Non-Autoregressive Machine Translation
Table 3: Test BLEU scores of non-autoregressive models using no refinement (# iterations = 1) and using refinement (# iterations = 10). Note that the # iterations = 1 results are purely non-autoregressive. ENGINE uses a CMLM as the inference network architecture and the transformer AR energy. The length beam size is 5 for CMLM and 3 for ENGINE.
['[EMPTY]', 'IWSLT14 DE-EN # iterations', 'IWSLT14 DE-EN # iterations', 'WMT16 RO-EN # iterations', 'WMT16 RO-EN # iterations']
S5SS0SSS0Px3 Impact of refinement iterations. Ghazvininejad et al. show improvements with multiple refinement iterations. Both improve with multiple iterations, though the improvement is much larger with CMLM. However, even with 10 iterations, ENGINE is comparable to CMLM on DE-EN and outperforms it on RO-EN.