Domain Adaptive Inference for Neural Machine Translation
Table 3: Test BLEU for es-en adaptive training. EWC reduces forgetting compared to other fine-tuning methods, while offering the greatest improvement on the new domain.
['[EMPTY]', '[BOLD] Training scheme', '[BOLD] Health', '[BOLD] Bio']
For es-en, the Health and Bio tasks overlap, but catastrophic forgetting still occurs under noreg (Table 3). Regularization reduces forgetting and allows further improvements on Bio over noreg fine-tuning. We find EWC outperforms the L2 approach