Distilled embedding: non-linear embedding factorization using knowledge distillation
Table 6: BLEU score for IWSLT Pt-En with compression rate 7.93x.
['Model', 'BLEU']
and we demonstrate through our experiments that at convergence most of the techniques are close in performance. The use of a non-linearity and adding embedding distillation also improves BLEU score after finetuning.