Adapting Multilingual Neural Machine Translation to Unseen Languages
Table 4: BLEU for ZST using models trained with different data-selection criteria. Pt+es and pt+es+it are the two varieties of the Select-fam method.
['[EMPTY]', 'Strategy', 'gl→en', 'en→gl']
In the gl→en direction, adding more data from related languages improves performance but the improvement slows down as more languages are added. Even without any test language data, performance increases from 13.59 BLEU for training only with pt (Select-one) to 24 BLEU for pt+es (Select-fam), while with it (pt+es+it) increases further by 1.34 BLEU. The MNMT model scores higher, but only by 1.77 BLEU when compared to best Select-fam strategy. Here, it is important to emphasize that: i) the MNMT model is trained using over 5M segments except for the test (gl-en) pair, meaning that the performance of Select-fam shows the possibility to improve a ZST by having more related languages but less data, ii) while the amount of data is the same for Select-one, Select-rand, and Select-pplx, the latter shows better performance, indicating the importance of the data selection criteria using perplexity. This gain highlights the importance of closely related languages for improving the performance on the LRL. However, the opposite direction, where we infer into unseen target language (gl), is a more challenging task that requires further investigation and the availability of at least monolingual data for the LRL.