Contrastive Analysis with Predictive Power: Typology Driven Estimation of Grammatical Error Distributions in ESL
Table 3: Results for prediction of relative error frequencies using the MAE metric across languages and error types, and the DKL metric averaged across languages. #Languages and #Mistakes denote the number of languages and grammatical error types on which a model outperforms Base.
['[EMPTY]', '[BOLD] Base', '[BOLD] NN', '[BOLD] Reg', '[BOLD] RegCA']
The baseline model Base sets the relative frequencies of the grammatical errors of a test language to the respective relative error frequencies in the training data. We also consider a stronger, language specific model called Nearest Neighbor (NN), which projects the error distribution of a target language from the typologically closest language in the training set, according to the cosine similarity measure. This baseline provides a performance improvement for the majority of the languages and error types, with an average error reduction of 13.3% on the MAE metric compared to Base, and improving from 0.052 to 0.046 on the KL divergence metric, thus emphasizing the general advantage of a native language adapted approach to ESL error prediction.