Approaching Neural Grammatical Error Correctionas a Low-Resource Machine Translation Task
Table 8: Shallow (Pretrain-Dec.) versus deep ensembles, with and without corresponding language models.
['Model', 'Dev', 'Prec.', 'Rec.', 'Test']
Both deep models improve significantly over the shallow model with the transformer model reaching our best result reported on the CoNLL 2014 test set. For that test set it seems that ensembling with language models that were used for pre-training is ineffective when measured with M2; while on the JFLEG data measured with GLEU we see strong improvements (Fig.