When and Why is Unsupervised Neural Machine Translation Useless?
Table 3: Unsupervised NMT performance where source and target training data are from different domains. The data size on both sides is the same (20M sentences).
['Domain ( [BOLD] en)', 'Domain ( [BOLD] de/ [BOLD] ru)', 'Bleu [%]  [BOLD] de-en', 'Bleu [%]  [BOLD] en-de', 'Bleu [%]  [BOLD] ru-en', 'Bleu [%]  [BOLD] en-ru']
(News Crawl). The results show that the domain matching is critical for unsupervised NMT. For instance, although German and English are very similar languages, we see the performance of German↔English deteriorate down to -11.8 % Bleu by the domain mismatch.