Multilingual Denoising Pre-training for Neural Machine Translation
Table 10: Unsupervised MT via Back-Translation. En-De, En-Ro are initialized by mBART02, while En-Ne, En-Si are initialized by mBART25. Our models are trained on monolingual data used in pre-training.
['[BOLD] Model', '[BOLD] Similar Pairs  [BOLD] En-De', '[BOLD] Similar Pairs  [BOLD] En-De', '[BOLD] Similar Pairs  [BOLD] En-Ro', '[BOLD] Similar Pairs  [BOLD] En-Ro', '[BOLD] Dissimilar Pairs  [BOLD] En-Ne', '[BOLD] Dissimilar Pairs  [BOLD] En-Ne', '[BOLD] Dissimilar Pairs  [BOLD] En-Si', '[BOLD] Dissimilar Pairs  [BOLD] En-Si']
Our models achieve large gains over non-pretrained models for all directions, and outperform XLM significantly for dissimilar pairs (En-Ne, En-Si) where the existing approaches completely fail. For similar pairs, our model also performs well against XLM and MASS, with the best numbers for En-X pairs.