Massively Multilingual Neural Machine Translation
Table 5: X→En test BLEU on the 103-language corpus
['[EMPTY]', 'Ar', 'Az', 'Be', 'De', 'He', 'It', 'Nl', 'Ro', 'Sk', 'Tr', 'Avg.']
First, we can see that both multilingual models perform better than the baselines in terms of average BLEU. This shows that massively multilingual many-to-many models can work well in realistic settings with millions of training examples, 102 languages and 204 jointly trained directions to-and-from English. Looking more closely, we note several different behaviors in comparison to the low-resource experiments on the TED Talks corpus. First, the many-to-one model here performs better than the many-to-many model. This shows that the previous result was indeed due to the pathologies of the low-resource dataset; when the training data is large enough and not multi-way-parallel there is no overfitting in the many-to-one model, and it outperforms the many-to-many model in most cases while they are trained identically.