Semantic Tagging with Deep Residual Networks
Table 4: Experiment results on Universal Dependencies (UD) test sets (% accuracy). Adding semtags as auxiliary tags results in the best results obtained so far on English UD datasets.
['[EMPTY]', 'Baselines MFC', 'Baselines TNT', 'Baselines Bi-lstm', 'Baselines Bi-gru', 'Basic CNN → [ITALIC] c', 'Basic CNN → [ITALIC] cbp', 'Basic CNN → [ITALIC] cbp∧→ [ITALIC] w', 'Basic CNN +aux [ITALIC] bp', 'ResNet → [ITALIC] c', 'ResNet → [ITALIC] c∧→ [ITALIC] w', 'ResNet +aux', 'ResNet → [ITALIC] cbp', 'ResNet → [ITALIC] cbp∧→ [ITALIC] w', 'ResNet +aux [ITALIC] bp']
We evaluate our system on v1.2 and v1.3 of the English part of the Universal Dependencies (UD) data. We report results for POS tagging alone, comparing to commonly used baselines and prior work using LSTMs, as well as using the fine-grained semantic tags as auxiliary information. For the +aux condition, we train a single joint model using a multi-task objective, with POS and ST as our two tasks. This model is trained on the concatenation of the ST silver data with the UD data, updating the loss of the respective task of an instance in each iteration. Hence, the weights leading to the UD softmax layer are not updated on the ST silver portion of the data, and vice-versa for the ST softmax layer on the UD portion of the data.