Explicit Representation of the Translation Space:Automatic Paraphrasing for Machine Translation Evaluation
Table 6: (WMT19 segment-level) Kendall’s τ correlations of Bleu with different paraphrase sets (size indicated in brackets) against segment-level human judgements. Results significantly better than sentenceBleu are indicated as follows: p≤0.05 (*), p≤0.01 (**), ≤0.001 (***).
['Approach', 'Method', 'de-en (85,365)', 'fi-en (38,307)', 'gu-en (31,139)', 'kk-en (27,094)', 'lt-en (21,862)', 'ru-en (46,172)', 'zh-en (31,070)', 'Ave.']
We again see small improvements over this baseline, although this is inconsistent across languages pairs and not significant. Again, there is little difference in the gains of the diverse methods, even with respect to using random cluster codes, which gave the least diversity. This suggests that the type of diversity being achieved is not successfully improving the metric as might be expected. The Sampled approach achieves amongst the greatest gains, as with system-level scores amongst the paraphrase approaches.