When and Why are Pre-trained Word Embeddings Useful for Neural Machine Translation?
(b) Pairwise comparison between two multilingual models
['multi:std here', 'multi:std 6/0', 'multi:pre+align on the', 'multi:pre+align 0/14']
We additionally performed pairwise comparisons between the top 10 n-grams that each system (selected from the task Gl → En) is better at generating, to further understand what kind of words pre-training is particularly helpful for. On the other hand, the improvements in systems without pre-trained embeddings were not very consistent, and largely focused on high-frequency words.