A Simple Baseline to Semi-Supervised Domain Adaptation for Machine Translation
Table 4: The comparison of three baselines: unadapted, copy, and back, between cases with and without pre-training when adapting them from the domain of LAW to MED and IT. Validation set BLEU scores are reported.
['[BOLD] Target', '[BOLD] Unadapted  [BOLD] w/', '[BOLD] Unadapted  [BOLD] w/o', '[BOLD] Copy  [BOLD] w/', '[BOLD] Copy  [BOLD] w/o', '[BOLD] Back  [BOLD] w/', '[BOLD] Back  [BOLD] w/o']
Through the experiments, we find that initializing the translation model with pre-trained parameters can not only benefit our method but also the baselines. Unadapted, Copy, Back, where we adapt from the LAW domain to MED and IT domains. For all three baselines, pre-training consistently brings in substantial improvements, although the pre-training process is performed via unsupervised language modeling training on Wikipedia text that is irrelevant to the target domain data we used here. This shows that proper initialization of models is crucial to the domain adaptation problem to circumvent the lack of supervised data in the target domain.