Context is Key: Grammatical Error Detection with Contextual Word Representations
Table 3: Error detection F0.5 of different embedding integration strategies (‘input’ vs. ‘output’) per model on all datasets.
['Flair', 'Input', 'Shared Task Dev 36.45', 'CoNLL Test 1 25.79', 'CoNLL Test 2 34.35', 'FCE test 49.97', 'JFLEG test 54.08']
We observe that, although performance varies across datasets and models, integration by concatenation to the word embeddings yields the best results across the majority of datasets for all models (BERT: 3/5 datasets; ELMo: 4/5 datasets; Flair: 5/5 datasets). The lower integration point allows the model to learn more levels of task-specific transformations on top of the contextual representations, leading to an overall better performance.