Toward Multilingual Neural Machine Translation with Universal Encoder and Decoder
Table 2: Results of the English→German system using large monolingual data.
['[BOLD] System', '[BOLD] tst2013 BLEU', '[BOLD] tst2013 ΔBLEU', '[BOLD] tst2014 BLEU', '[BOLD] tst2014 ΔBLEU']
The first result is not encouraging when its performance is even worse than the baseline NMT which is trained on the small parallel data only. Not using the same information in the source side, as we discussed in case of mix-multi-source strategy, could explain the degrading in performance of such a system. But we believe that the magnitude and unbalancing of the corpus are the main reasons. The data contains nearly four millions sentences but only around twenty thousands of them (0.5%) are the genuine parallel data. As a quick attempt, after we get the model with that big data, we continue training on the real parallel corpus for some more epochs.