Machine Translation Evaluation with Neural Networks
Table 5: Kendall’s tau (τ) on WMT12 for alternative cost functions using 4metrics+syntax25+Wiki-GW50.
['[BOLD] Details', '[BOLD] Kendall’s  [ITALIC] τ  [BOLD] cz', '[BOLD] Kendall’s  [ITALIC] τ  [BOLD] de', '[BOLD] Kendall’s  [ITALIC] τ  [BOLD] es', '[BOLD] Kendall’s  [ITALIC] τ  [BOLD] fr', '[BOLD] Kendall’s  [ITALIC] τ  [BOLD] AVG']
our Kendall cost. We can see that using the Kendall cost enables effective learning, although it is eventually outperformed by the logistic cost. Our investigation revealed that this was due to a combination of slower convergence and poor initialization. Therefore, we further experimented with a setup where we first used the logistic cost to pre-train the neural network, and then we switched to the Kendall cost in order to perform some finer tuning.