Auxiliary Objectives for Neural Error Detection Models
Table 4: Results on error detection when the model is pre-trained on different tasks.
['Aux dataset', 'FCE TEST', 'CoNLL-14 TEST1', 'CoNLL-14 TEST2']
In the first setting, each of these datasets is used to train a sequence labeling model for their respective tasks, and the resulting model is used to initialise a network for training an error detection system. While it is common to preload word embeddings from a different model, this strategy extends the idea to the compositional components of the network. One of the main advantages of multi-task learning is regularisation, actively encouraging the model to learn more general-purpose features, something which is not exploited in this setting since the training happens in separate stages.