Trivial Transfer Learning for Low-Resource Neural Machine Translation
Table 4: Results of child following a parent with swapped direction. “Baseline” is child-only training. “Aligned” is the more natural setup with English appearing on the “correct” side of the parent, the numbers in this column thus correspond to those in Table 2.
['Parent - Child', 'Transfer', 'Baseline', 'Aligned']
This interesting result should be studied in more detail. \newcitefirat-cho-bengio:etal:2016 hinted possible gains even when both languages are distinct from the low-resource languages but in a multilingual setting. Not surprisingly, the improvements are better when the common language is aligned. We see gains in both directions, although not always statistically significant. Future work should investigate if this performance boost is possible even for high-resource languages. Similar behavior has been shown in \newciteniu-denkowski-carpuat:2018:WNMT2018, where in contrast to our work they mixed the data together and added an artificial token indicating the target language.