Explicit Representation of the Translation Space:Automatic Paraphrasing for Machine Translation Evaluation
Table 7: Correlations on the 500-sentence subset.
['[EMPTY]', 'Method', 'Correlation System', 'Correlation Segment']
We report correlations for the 500-sentence WMT19 de-en subset with human results in Tab. Although a small subset, a few observations can be made. Firstly we see a lot of variance, both across and within methods. Most importantly, the manual paraphrases do not achieve higher results than the other methods, including the baselines. The method with the highest correlation on this subset is again Sampled, which reinforces the previous observations that adequacy is not necessarily a defining factor; even if the paraphrases are of good quality (i.e. produced by humans), these paraphrases will not necessarily help Bleu’s performance. As with the main results tables, here the constraints-based approach tends to help slightly more with segment-level correlations, suggesting that it could be slightly more beneficial to target areas of the translation space for sentenceBleu than to produce as diverse reference as possible.