Deep Semantic Role Labeling with Self-Attention
Table 1: Comparison with previous methods on the CoNLL-2005 dataset. We report the results in terms of precision (P), recall (R), F1 and percentage of completely correct predicates (Comp.). Our single and ensemble model lead to substantial improvements over the previous state-of-the-art results.
['Model', 'Development P', 'Development R', 'Development F1', 'Development Comp.', 'WSJ Test P', 'WSJ Test R', 'WSJ Test F1', 'WSJ Test Comp.', 'Brown Test P', 'Brown Test R', 'Brown Test F1', 'Brown Test Comp.', 'Combined F1']
On the CoNLL-2005 dataset, the single model of DeepAtt with RNN, CNN and FFN nonlinear sub-layers achieves an F1 score of 82.3, 82.3 and 83.4 respectively. The FFN variant outperforms previous best performance by 1.8 F1 score. Remarkably, we get 74.1 F1 score on the out-of-domain dataset, which outperforms the previous state-of-the-art system by 2.0 F1 score. On the CoNLL-2012 dataset, the single model of FFN variant also outperforms the previous state-of-the-art by 1.0 F1 score. When ensembling 5 models with FFN nonlinear sub-layers, our approach achieves an F1 score of 84.6 and 83.9 on the two datasets respectively, which has an absolute improvement of 1.4 and 0.5 over the previous state-of-the-art. These results are consistent with our intuition that the self-attention layers is helpful to capture structural information and long distance dependencies.