An Empirical Study of Incorporating Pseudo Datainto Grammatical Error Correction
Table 5: Comparison of our best model and current top models: a bold value indicates the best result within the column.
['Model', 'Ensemble', 'CoNLL-2014 ( [ITALIC] M2 scorer) Prec.', 'CoNLL-2014 ( [ITALIC] M2 scorer) Rec.', 'CoNLL-2014 ( [ITALIC] M2 scorer) F0.5', 'CoNLL-2014 (ERRANT) Prec.', 'CoNLL-2014 (ERRANT) Rec.', 'CoNLL-2014 (ERRANT) F0.5', 'JFLEG GLEU', 'BEA-test (ERRANT) Prec.', 'BEA-test (ERRANT) Rec.', 'BEA-test (ERRANT) F0.5']
We evaluate the performance of PretLarge on test sets and compare the scores with the current top models. This result outperforms not only all previous single-model results but also all ensemble results except for that by Grundkiewicz et al. In addition, PretLarge+SSE+R2L+SED can further improve the performance on BEA-test (F0.5=70.2). However, unfortunately, incorporating SED decreased the performance on CoNLL-2014 and JFLEG. This fact implies that SED is sensitive to the domain of the test set since the SED model is fine-tuned with the official validation split of BEA dataset. We leave this sensitivity issue as our future work.