Machine Translation Evaluation with Neural Networks
Table 10: Comparing NNRK to the state of the art at the system level on the WMT12 dataset, translating into English.
['[BOLD] System', '[BOLD] Details', '[BOLD] Spearman’s rank correlation  [ITALIC] ρ  [BOLD] cz ', '[BOLD] Spearman’s rank correlation  [ITALIC] ρ  [BOLD] de', '[BOLD] Spearman’s rank correlation  [ITALIC] ρ  [BOLD] es', '[BOLD] Spearman’s rank correlation  [ITALIC] ρ  [BOLD] fr', '[BOLD] Spearman’s rank correlation  [ITALIC] ρ  [BOLD] AVG']
Not surprisingly, they are on par with what we saw at the segment level. Once again, our NNRK metric outperformed the metrics from the WMT12 competition, as well as the best post-competition result of DiscoTK; moreover, our NNRK metric is strong across all language pairs. Also, as expected, the system-level score based on the NNRKMean segment-level score performed better than the one based on NNRKZero.