Domain Adaptive Inference for Neural Machine Translation
Table 4: Test BLEU for en-de adaptive training, with sequential adaptation to a third task. EWC-tuned models give the best performance on each domain.
['[EMPTY]', '[BOLD] Training scheme', '[BOLD] News', '[BOLD] TED', '[BOLD] IT']
In the en-de News/TED task (Table 4), all fine-tuning schemes give similar improvements on TED. However, EWC outperforms no-reg and L2 on News, not only reducing forgetting but giving 0.5 BLEU improvement over the baseline News model. [CONTINUE] The IT task is very small: training on IT data alone results in over-fitting, with a 17 BLEU improvement under fine-tuning. [CONTINUE] fine-tuning rapidly forgets previous tasks. EWC reduces forgetting on two previous tasks while further improving on the target domain.