Non-Parametric Adaptation for Neural Machine Translation
Table 7: Comparison of test translation quality (BLEU) with different adaptation strategies. The base model (Transformer Base) is trained on the WMT dataset. Fine-tuning corresponds to fine-tuning based adaptation, where we initialize the domain-specific model from the WMT pre-trained Base model, and fine-tune it on the in-domain dataset for a few epochs. Non-parametric corresponds to our semi-parametric NMT model, adapted to in-domain data during inference by retrieving neighbors from the in-domain training corpus.
['Adaptation Strategy', 'newstest 14', 'IWSLT 2015', 'OpenSub', 'JRC-Acquis']
We notice that the non-parametric adaptation strategy significantly out-performs the base model on all 4 datasets. More importantly, the we find that our approach is capable of adapting to both, JRC-Acquis and OpenSubtitles, via just the retrieval apparatus, and out-performs the fully fine-tuned model indicating that non-parametric adaptation might be a reasonable approach when adapting to a lot of narrow domains or documents.