Multilingual Neural Machine Translation with Knowledge Distillation
Table 6: BLEU scores of selective distillation (our method) and distillation all the time during the training process on the Ted talk dataset.
['[EMPTY]', 'Bg', 'Et', 'Fi', 'Fr', 'Gl', 'Hi', 'Hy', 'Ka']
Selective Distillation We list the 16 languages on which the two methods (selective distillation, and distillation all the time) that have difference bigger than 0.5 in terms of BLEU score. It can be seen that selective distillation performs better on 13 out of 16 languages, with large BLEU score improvements, which demonstrates the effectiveness of the selective distillation.