Regularizing Neural Machine Translation by Target-bidirectional Agreement
Table 3: Case-sensitive BLEU scores (%) for English-German and Chinese-English translation on WMT test sets. Edinburgh [Sennrich et al.2017] and Sogou [Wang et al.2017b] NMT systems are No.1 system in leaderboard of WMT 2017’s English-German and Chinese-English translation tasks respectively.
['System', 'English-German newstest2016', 'English-German newstest2017', 'Chinese-English newsdev2017', 'Chinese-English newstest2017']
Since back-translation method brings in more synthetic data, we choose Transformer-big settings defined in \citeauthorVaswani2017AttentionIA \shortciteVaswani2017AttentionIA for this setting. -Sx4SSx3SSS0Px1 Only Parallel Corpora. It confirms the effectiveness of leveraging agreements between L2R and R2L models. Additionally, our approach significantly outperforms Transformer+JS, yielding the best BLEU score on English-German and Chinese-English test sets respectively when merely using bilingual data. These results further prove the effectiveness of our method. When back-translation method is employed, Transformer-big+BT achieves 1.21 and 1.5 BLEU point improvements over Transformer-big on English-German and Chinese-English sets respectively. Our method can further gain remarkable improvements based on back-translation method, resulting in the best results on all translation tasks. These results show that in semi-supervised learning scenarios, the NMT model still can benefit from our proposed approach. In addition, our single model Transformer-big+BT+RT even achieves the best performance on WMT17’s English-German and Chinese-English translation tasks, among all the reported results, including ensemble systems.