There’s No Comparison: Reference-less Evaluation Metricsin Grammatical Error Correction
Table 1: Correlation between the human and metric rankings.
['[BOLD] Metric', '[BOLD] Spearman’s  [ITALIC] ρ', '[BOLD] Pearson’s  [ITALIC] r']
The error-count metrics, ER and LT, have stronger correlation than all RBMs except for GLEU, which is the current state of the art. GLEU has the strongest correlation with the human ranking (ρ=0.852, r=0.838), followed closely by ER, which has slightly lower Pearson correlation ( r=0.829) but equally as strong rank correlation, which is arguably more important when comparing different systems. I-measure and LFM have similar strength correlations, and M2 is the lowest performing metric, even though its correlation is still strong (ρ=0.648, r=0.641).