Machine Translation Evaluation with Neural Networks
Table 11: Comparing to the state of the art at the segment level on the WMT14 dataset, translating into English.
['[BOLD] System', '[BOLD] Details', '[BOLD] Kendall’s  [ITALIC] τ (WMT14-style)  [BOLD] fr ', '[BOLD] Kendall’s  [ITALIC] τ (WMT14-style)  [BOLD] de', '[BOLD] Kendall’s  [ITALIC] τ (WMT14-style)  [BOLD] hi', '[BOLD] Kendall’s  [ITALIC] τ (WMT14-style)  [BOLD] cs', '[BOLD] Kendall’s  [ITALIC] τ (WMT14-style)  [BOLD] ru', '[BOLD] Kendall’s  [ITALIC] τ (WMT14-style)  [BOLD] AVG']
Once again, our NNRK metric is very competitive and outperforms all rivals, except for the tuned DiscoTK, which combines about twenty strong pre-existing evaluation metrics (in addition to discourse-based kernels), while we only incorporate four pre-existing metrics. ’s τ points absolute; we are also better on all four language pairs. Overall, we are best on Hindi-English, but we are surprisingly weak on Russian-English.