Massively Multilingual Neural Machine Translation
Table 7: Supervised performance while varying the number of languages involved
['[EMPTY]', 'Ar-En', 'En-Ar', 'Fr-En', 'En-Fr', 'Ru-En', 'En-Ru', 'Uk-En', 'En-Uk', 'Avg.']
We can see that in most cases the best results are obtained using the 5-to-5 model, showing that there is indeed a trade off between the number of languages and translation accuracy when using a fixed model capacity and the same training setup. One may expect that the gaps between the different models should become smaller and even close with more updates, as the models with more languages see less examples per language in each batch, thus requiring more updates to improve in terms of BLEU. However, in our setting these gaps did not close even after the models converged, leaving 2.73 average BLEU difference between the 5-to-5 and the 103-to-103 model.