Using Monolingual Data in Neural Machine Translation:a Systematic Study
Table 6: Performance wrt. different GAN setups
['[BOLD] English→French', '[BOLD] English→French  [BOLD] test-07', '[BOLD] English→French  [BOLD] test-07', '[BOLD] English→French  [BOLD] test-07', '[BOLD] English→French  [BOLD] test-08', '[BOLD] English→French  [BOLD] test-08', '[BOLD] English→French  [BOLD] test-08', '[BOLD] English→French  [BOLD] newstest-14', '[BOLD] English→French  [BOLD] newstest-14', '[BOLD] English→French  [BOLD] newstest-14']
On top of the copy-marked setup, our GANs do not provide any improvement in both language pairs, with the exception of a small improvement for English-French on the out-of-domain test, which we understand as a sign that the model is more robust to domain variations, just like when adding pseudo-source noise. When combined with noise, the French model yields the best performance we could obtain with stupid BT on the in-domain tests, at least in terms of BLEU and BEER. On the News domain, we remain close to the baseline level, with slight improvements in German. Finally note that the GAN architecture has two differences with basic copy-marked: (a) a distinct encoder for real and pseudo-sentence; (b) a different training regime for these encoders. To sort out the effects of (a) and (b), we reproduce the GAN setup with BT sentences, instead of copies. : translation degrades in French for all metrics. Adding GANs on top of the pseudo-encoder was not able to make up for the degradation observed in French, but allowed the German system to slightly outperform backtrans-nmt. Even though this setup is unrealistic and overly costly, it shows that GANs are actually helping even good systems.