Machine Translation Evaluation Using Bi-directional Entailment
Table 2: Pearson’s System Correlation on WMT’17 data for baseline and participating metrics. Bold results with symbol † denotes highest correlation and symbol †† denotes second highest correlation for a language pair.
['[BOLD] Metric/Model', '[BOLD] cs-en', '[BOLD] de-en', '[BOLD] fi-en', '[BOLD] lv-en', '[BOLD] ru-en', '[BOLD] tr-en', '[BOLD] zh-en', '[BOLD] Average']
Our metric Bi-Di-Ent performs better than BLEU for all language pairs except for lv-en and tr-en pair. The correlation scores are also comparable to the top performing evaluation metrics. It achieves best scores in de-en and ru-en language pair among all metrics. Overall, our metric has the second highest average correlation score of 0.956, highest being 0.960.