Massively Multilingual Neural Machine Translation in the Wild: Findings and Challenges
Table 4: Effect of increasing the number of languages on the zero-shot performance of multilingual models.
['[EMPTY]', '[ITALIC] De→ [ITALIC] Fr', '[ITALIC] Be→ [ITALIC] Ru', '[ITALIC] Yi→ [ITALIC] De', '[ITALIC] Fr→ [ITALIC] Zh', '[ITALIC] Hi→ [ITALIC] Fi', '[ITALIC] Ru→ [ITALIC] Fi']
We observe that zero-shot performance on similar languages, in this case Be→Ru and Yi→De , is extremely high. We also notice that the zero-shot performance for most language pairs increases as we move from the 10 language model to the 102 language model, possibly due to the regularization effect in the capacity constrained setting, similar to what was observed in Aharoni et al. This also indicates why methods that explicitly force languages to share the same representation space Arivazhagan et al. ; Gu et al.