Using Monolingual Data in Neural Machine Translation:a Systematic Study
Table 7: Deep-fusion model
['[BOLD] English→French', '[BOLD] English→French  [BOLD] test-07', '[BOLD] English→French  [BOLD] test-07', '[BOLD] English→French  [BOLD] test-07', '[BOLD] English→French  [BOLD] test-08', '[BOLD] English→French  [BOLD] test-08', '[BOLD] English→French  [BOLD] test-08', '[BOLD] English→French  [BOLD] newstest-14', '[BOLD] English→French  [BOLD] newstest-14', '[BOLD] English→French  [BOLD] newstest-14']
They show that deep-fusion hardly improves the Europarl results, while we obtain about +0.6 BLEU over the baseline on newstest-2014 for both languages. deep-fusion differs from stupid BT in that the model is not directly optimized on the in-domain data, but uses the LM trained on Europarl to maximize the likelihood of the out-of-domain training data. Therefore, no specific improvement is to be expected in terms of domain adaptation, and the performance increases in the more general domain. Combining deep-fusion and copy-marked + noise + GANs brings slight improvements on the German in-domain test sets, and performance out of the domain remains near the baseline level.