Source Dependency-Aware Transformer with Supervised Self-Attention
Table 3: BLEU scores (%) for Chinese-to-English (Zh-En), English-to-Chinese (En-Zh) translation on WMT2017 datasets and English-to-German (En-De) task. Both char-level BLEU (CBLEU) and word-level BLEU (WBLEU) are used as metrics for the En-Zh task.
['System', 'Zh-En', 'En-Zh CBLEU', 'En-Zh WBLEU', 'En-De']
To verify the effect of syntax knowledge on large-scale translation tasks, we further conduct three experiments on the WMT2017 bidirectional English-Chinese tasks and WMT2014 English-to-German. For the Chinese-to-English, our proposed method outperforms baseline by 0.86 BLEU score. For the English-to-Chinese task, the Transformer+CSH+PSH gains 0.91 and 1.05 improvements on char-level BLEU and word-level BLEU respectively. For En-De task, the improvement is 0.6 which is not as much as the other two. We speculate that as the grammars of English and German are very similar, the original model can capture the syntactic knowledge well. Even though, the improvement still illustrates the effectiveness of our method.