Improving Unsupervised Word-by-Word Translationwith Language Model and Denoising Autoencoder
Table 1: Translation results on German↔English newstest2016 and French↔English newstest2014.
['[EMPTY]', 'de-en', 'en-de', 'fr-en', 'en-fr']
LM improves word-by-word baselines consistently in all four tasks, giving at least +3% Bleu. When our denoising model is applied on top of it, we have additional gain around +3% Bleu. Note that our methods do not involve any decoding steps to generate pseudo-parallel training data, but still perform better than unsupervised MT systems that rely on repetitive back-translations Artetxe et al. ; Lample et al. The total training time of our method is only 1-2 days with a single GPU.