A Neural Grammatical Error Correction System Built On Better Pre-training and Sequential Transfer Learning
Table 10: Results on error types.
['[BOLD] Error types', '[BOLD] P', '[BOLD] R', '[BOLD] F0.5']
Here, we give an analysis of our model’s performance on some of the major ERRANT error categories on the W&I test set. We observe that our model performs well on syntax relevant error types, i.e., subject-verb agreement (VERB:SVA) (84.09 F0.5), Our model also achieves 77.26 on spelling errors (SPELL) and 75.83 on orthographic errors (ORTH), both of which are improvements made mostly by our context-aware neural spellchecker. Our model also achieves 77.86 on punctuation errors (PUNCT), which happen to be the most common error category in the W&I+L dataset. This may be due to both our use of extracted errors from the W&I dataset during pre-training and our fine-tuning step. Finally, we find it challenging to match human annotators’ “naturalness” edits, such as VERB (26.76), NOUN (41.67), and OTHER (36.53). This is possibly due to the variability in annotation styles and a lack of large training data with multiple human annotations.