Using Monolingual Data in Neural Machine Translation:a Systematic Study
Table 8: BLEU scores with selective parameter freezing
['[EMPTY]', '[BOLD] English→French  [BOLD] test-07', '[BOLD] English→French  [BOLD] test-08', '[BOLD] English→French  [BOLD] nt-14', '[BOLD] English→German  [BOLD] test-07', '[BOLD] English→German  [BOLD] test-08', '[BOLD] English→German  [BOLD] nt-14']
The backtrans-nmt setup is hardly impacted by selective updates: updating the only decoder leads to a degradation of at most 0.2 BLEU. For copy-marked, we were not able to freeze the source embeddings, since these are initialized when fine-tuning begins and therefore need to be trained. We observe that freezing the encoder and/or the attention parameters has no impact on the English-German system, whereas it slightly degrades the English-French one. This suggests that using artificial sources, even of the poorest quality, has a positive impact on all the components of the network, which makes another big difference with the LM integration scenario.