BLEU might be Guilty but References are not Innocent
Table 10: Measuring the degree of translationese, reporting percent difference wrt. to de.orig. Higher lexical variety, lexical density, and length variety imply less translationese sentences. Values at or exceeding those of natural text are bolded.
['[EMPTY]', 'Lex. Var.', 'Lex. Density', 'Len. Var.']
For all metrics, de.tr gets the lowest scores, confirming that standard human translations yield more translationese style output. The paraphrases, on the other hand, have lexical density and length variety that is much higher than both the translated sentences and the original German sentences, though they have a lower lexical variety. This demonstrates that we were able to remove many of the translationese artifacts by paraphrasing as much as possible.