Human Parity in Machine Translation According to Humans
Table 1: BLEU scores between pairs of three translations (Ref, Human and the best MT system) for German→English at the news translation task of WMT 2019.
['MT, Ref', 'MT, Human', 'Ref, Human']
The claim of parity for German→English results therefore from the fact that Human and the output of an MT system (Facebook-FAIR) were compared separately to a gold standard translation, Ref, and the overall ratings that they obtained were not significantly different from each other. If there was reference bias in this case, it could be that Human was penalised for being different than Ref.