Deep Semantic Role Labeling with Self-Attention
Table 2: Experimental results on the CoNLL-2012 dataset. The metrics are the same as above. Again, our model achieves the state-of-the-art performance.
['Model', 'Development P', 'Development R', 'Development F1', 'Development Comp.', 'Test P', 'Test R', 'Test F1', 'Test Comp.']
On the CoNLL-2005 dataset, the single model of DeepAtt with RNN, CNN and FFN nonlinear sub-layers achieves an F1 score of 82.3, 82.3 and 83.4 respectively. The FFN variant outperforms previous best performance by 1.8 F1 score. Remarkably, we get 74.1 F1 score on the out-of-domain dataset, which outperforms the previous state-of-the-art system by 2.0 F1 score. On the CoNLL-2012 dataset, the single model of FFN variant also outperforms the previous state-of-the-art by 1.0 F1 score. When ensembling 5 models with FFN nonlinear sub-layers, our approach achieves an F1 score of 84.6 and 83.9 on the two datasets respectively, which has an absolute improvement of 1.4 and 0.5 over the previous state-of-the-art. These results are consistent with our intuition that the self-attention layers is helpful to capture structural information and long distance dependencies.