When and Why are Pre-trained Word Embeddings Useful for Neural Machine Translation?
Table 2: Effect of pre-training on BLEU score over six languages. The systems use either random initialization (std) or pre-training (pre) on both the source and target sides.
['[BOLD] Src → →  [BOLD] Trg', 'std', 'pre std', 'std pre', 'pre']
Comparing the second and third columns, we can see the increase is much more significant with pre-trained source language embeddings. This indicates that the majority of the gain from pre-trained word embeddings results from a better encoding of the source sentence.