Multilingual Denoising Pre-training for Neural Machine Translation
Table 4: Comparison with Other Pre-training Approaches on WMT16 Ro-En.
['[BOLD] Pre-training  [BOLD] Model', '[BOLD] Pre-training  [BOLD] Data', '[BOLD] Fine-tuning  [BOLD] En→Ro', '[BOLD] Fine-tuning  [BOLD] Ro→En', '[BOLD] Fine-tuning  [BOLD] +BT']
We consider En-Ro translation, the only pair with established results. Our mBART model outperforms all the other pre-trained models, both with and without BT augmentation. We also show comparisons with the conventional BART model trained on the same En and Ro data only. Both have improvements over baselines, while worse than mBART results, indicating pre-training in a multilingual setting is essential. Moreover, combining BT leads to additional gains, resulting in a new state-of-the-art for Ro-En translation.