Leveraging Monolingual Data with Self-Supervisionfor Multilingual Neural Machine Translation
Table 3: Translation quality of the new language added to Multilingual NMT using just monolingual data. Multilingual NMT here is a multilingual model with 30 language pairs, Mono. Only is a bilingual model used as a baseline trained with only monolingual data with self-supervised learning, Multilingual NMT-xx is a multilingual model trained on 28 language pairs (xx is the language not present in the model). Multilingual NMT-xx + Mono. is a multilingual model with 28 language pairs but only monolingual data for xx.
['[EMPTY]', 'fr_en', 'en_fr', 'de_en', 'en_de', 'ro_en', 'en_ro', 'lt_en', 'en_lt', 'lv_en', 'en_lv', 'hi_en', 'en_hi']
We find that simply adding monolingual data for a new language to the training procedure of a multilingual model is sufficient to obtain strong translation quality for several languages, often attaining within a few BLEU points of the fully supervised multilingual baseline, without the need for iterative back-translation. We also notice significant quality improvements over models trained with just self-supervised learning using monolingual data for a variety of languages. et al. This suggests that jump-starting the iterative back-translation process from multilingual models might be a promising avenue to supporting new languages.