Domain Differential Adaptation for Neural Machine Translation
Table 2: Performance of DDA-Deep when fusing different parts of models on the law and medical datasets.
['[BOLD] Components', '[BOLD] LAW-MED', '[BOLD] MED-LAW']
We find that generally fusing hidden states is better than fusing word embeddings, and fusing hidden states together with word embeddings does not show any improvements over simply fusing hidden states alone. These results indicate that combining the higher-level information captured by the encoder states is more advantageous for domain adaptation. Also, we found that directly using DDA-Deep to fuse output probabilities was unstable even after trying several normalization techniques, possibly because of the sensitivity of output probabilities.