Trivial Transfer Learning for Low-Resource Neural Machine Translation
Table 2: Transfer learning with English reused either in source (encoder) or target (decoder). The column “Transfer” is our method, baselines correspond to training on one of the corpora only. Scores (BLEU) are always for the child language pair and they are comparable only within lines or when the child language pair is the same. “Unrelated” language pairs in bold. Upper part: parent larger, lower part: child larger. (“EN” lowercased just to stand out.)
['Parent - Child', 'Transfer', 'Baselines: Only Child', 'Baselines: Only Parent']
Furthermore, the improvement is not restricted only to related languages as Estonian and Finnish as shown in previous works. We reach an improvement of 3.38 BLEU for ENET when parent model was ENCS, compared to improvement of 2.71 from ENFI parent. This statistically significant improvement contradicts \newcitedabre2017empirical who concluded that the more related the languages are, the better transfer learning works. We see it as an indication that the size of the parent training set is more important than relatedness of languages. On the other hand, this transfer learning works well only when the parent has more training data than the child. This is another indication, that the most important is the size of the parent corpus compared to the child one.