Transfer Learning for Low-Resource Neural Machine Translation
Table 1: NMT models with attention are outperformed by standard string-to-tree statistical MT (SBMT) when translating low-resource languages into English. Train/test bitext corpus sizes are given in word tokens on the English side. Single-reference, case-insensitive Bleu scores are given for held-out test corpora.
['Language', 'Train', 'Test', 'SBMT', 'NMT']
Neural machine translation (NMT) NMT systems have achieved competitive accuracy rates under large-data training conditions for language pairs such as English-French. However, neural methods are data-hungry and learn poorly from low-count events. This behavior makes vanilla NMT a poor choice for low-resource languages, where parallel data is scarce. For all of our experiments with low-resource languages we use French as the parent source language and for child source languages we use Hausa, Turkish, Uzbek, and Urdu. The target language is always English. For comparison, our parent French-English model uses a training set with 300 million English tokens and achieves 26 Bleu on the development set. There is a large gap between the SBMT and NMT systems without using our transfer method.