APE at Scale and its Implications on MT Evaluation Biases
Table 9: Average Bleu scores for WMT18 English→German newstest2014-2017. We run our APE model a second time on the output of the already post-editied output.
['[EMPTY]', '[BOLD] average orig-de', '[BOLD] average orig-en']
We can apply our APE model in an iterative fashion several times on the same output. As a result, 75% of the sentences did not change. The remaining sentences lowered the Bleu scores on average by 0.1 points for German-original half and by 0.7 points for the English-original half of the test sets.