RETURNN as a Generic Flexible Neural Toolkit with Application to Translation and Speech Recognition
Table 4: Performance comparison on WMT 2017 English→German. The baseline systems (upper half) are trained on the parallel data of the WMT Enlgish→German 2017 task. We downloaded the hypotheses from here.66footnotemark: 6 The WMT 2017 system hypotheses (lower half) are generated using systems having additional back- translation (bt) data. These hypotheses are downloaded from here.77footnotemark: 7
['System', 'BLEU [%]']
We observe that our toolkit outperforms all other toolkits. The best result obtained by other toolkits is using Marian (25.5% BLEU). In comparison, RETURNN achieves 26.1%. We also compare RETURNN to the best performing single systems of WMT 2017. In comparison to the fine-tuned evaluation systems that also include back-translated data, our model performs worse by only 0.3 to 0.9 BLEU. We did not run experiments with back-translated data, which can potentially boost the performance by several BLEU points.