Transfer Learning for Low-Resource Neural Machine Translation
Table 8: Transfer with parent models trained only on English data. The child data is the Uzbek-English corpus from Table 3. The English-English parent learns to copy English sentences, and the EngPerm-English learns to un-permute scrambled English sentences. The LM is a 2-layer LSTM RNN language model trained on the English corpus.
['Setting', 'Bleu', 'PPL']
In the above experiments, we use a parent model trained on a large French/English bilingual corpus. One might hypothesize that our gains come from exploiting the English half of the corpus as an additional language model resource. Therefore, we explore transfer learning for the child model with parent models that only use the English side of the bilingual corpus. Additionally, we train a parent model that is just an RNN language model. These results show that our transfer learning is not simply importing an English language model, but making use of translation parameters learned from the parent’s large bilingual text.