Linguistically-Informed Self-Attention for Semantic Role Labeling
Table 6: Average SRL F1 on CoNLL-2005 for sentences where LISA (L) and D&M (D) parses were completely correct (+) or incorrect (–).
['[EMPTY]', 'L+/D+', 'L–/D+', 'L+/D–', 'L–/D–']
First we assess SRL F1 on sentences divided by parse accuracy. Both parsers are correct on 26% of sentences. Here there is little difference between any of the models, with LISA models tending to perform slightly better than SA. Both parsers make mistakes on the majority of sentences (57%), difficult sentences where SA also performs the worst. These examples are likely where gold and D&M parses improve the most over other models in overall F1: Though both parsers fail to correctly parse the entire sentence, the D&M parser is less wrong ( 87.5 vs. 85.7 average LAS), leading to higher SRL F1 by about 1.5 average F1.