Meta-Embeddings Based On Self-Attention
Table 5: Machine translation accuracy in terms of BLEU for WMT En-De and WMT En-Fr on newstest2014.
['[BOLD] Model', '[BOLD] Param', '[BOLD] WMT En-De', '[BOLD] WMT En-Fr']
Specifically, our model is able to achieve a state-of-the-art score on WMT 2014 En-De benchmark, and still competitive in WMT 2014 English to France bench mark. Worth mentioning, the meta-embedding Duo Transformer has outperformed the vanilla transformer by 1.3 and 1.1 BLEU score on each task, further proving the advantage of the meta-embedding mechanism.