Domain Adaptive Inference for Neural Machine Translation
Table 6: Test BLEU for 2-model es-en and 3-model en-de model ensembling for models adapted with EWC, compared to oracle model last trained on each domain, chosen if test domain is known. BI+IS outperforms uniform ensembling and in some cases outperforms the oracle.
['[BOLD] Decoder configuration', '[BOLD] es-en  [BOLD] Health', '[BOLD] es-en  [BOLD] Bio', '[BOLD] en-de  [BOLD] News', '[BOLD] en-de  [BOLD] TED', '[BOLD] en-de  [BOLD] IT']
In Table 6 we apply the best adaptive decoding scheme, BI+IS, to models fine-tuned with EWC. [CONTINUE] EWC models perform well over multiple domains, so the improvement over uniform ensembling is less striking than for unadapted models. Nevertheless adaptive decoding improves over both uniform ensembling and the oracle model in most cases.