Transfer Learning for Low-Resource Neural Machine Translation
Table 3: Our transfer method applied to re-scoring output n-best lists from the SBMT system. Additionally, the ‘LM’ column shows the results when an RNN LM was trained on the large English corpus and used to re-score the n-best list.
['Setting', 'SBMT', 'NMT', 'Xfer', 'LM']
We also use the NMT model with transfer learning to re-score output n-best lists (n=1000) from the SBMT system. Transfer NMT models give the highest gains over using re-scoring with a neural language model or an NMT system that does not use transfer. The neural language model is an LSTM RNN with 2 layers and 1000 hidden states. It has a target vocabulary of 100K and is trained using noise-contrastive estimation From re-scoring with the transfer model, we get an improvement of 1.1–1.6 Bleu points above the strong SBMT system.