A Nested Attention Neural Hybrid Model for Grammatical Error Correction
Table 3: F0.5 results on the CoNLL-13 and CoNLL-14 test sets of main model architectures.
['[BOLD] Model', '[BOLD] Performance Dev', '[BOLD] Performance Test']
In addition to the word-level baseline, we include the performance of a hybrid model with a single level of attention, which follows the work of \newciteluong-manning:2016:P16-1 for machine translation, and is the first application of a hybrid word/character-level model to grammatical error correction. Based on hyper-parameter selection, the character-level component weight of the loss is α=1 for the basic hybrid model. N16-1 (38.77 versus 39.9). We attribute the difference to differences in the training set and the word-alignment methods used. Our reimplementation serves to provide a controlled experimental evaluation of the impact of hybrid models and nested attention on the GEC task. As seen, our nested attention hybrid model substantially improves upon the baseline, achieving a gain of close to 3 points on the test set. The hybrid word/character model with a single level of attention brings a large improvement as well, showing the importance of character-level information for this task. We delve deeper into the impact of nested attention for the hybrid model in Section 5.