Auxiliary Objectives for Neural Error Detection Models
Table 5: Results on error detection when training is alternated between the two tasks (e.g., error detection and POS tagging) and datasets.
['Aux dataset', 'FCE TEST', 'CoNLL-14 TEST1', 'CoNLL-14 TEST2']
In the second set of experiments, we explore the possibility of training on the second domain and task at the same time as error detection. Similar to Collobert and Weston By alternating between the two tasks, the model is able to retain the regularisation benefits. One possible explanation is that the domain and writing style of these auxiliary datasets is very different from the learner writing corpus, and the model ends up optimising in an unnecessary direction.