Jointly Predicting Predicates and Argumentsin Neural Semantic Role Labeling
Table 3: Comparison on the CoNLL 05 development set against previous systems in terms of unlabeled agreement with gold constituency (Syn%) and each type of SRL-constraints violations (Unique core roles, Continuation roles and Reference roles).
['Model/Oracle', 'SRL F1', 'Syn %', 'SRL-Violations U', 'SRL-Violations C', 'SRL-Violations R']
For example, on the CoNLL 2005 test set, our model has lower complete-predicate accuracy (62.6%) than the BIO systems He et al. ; Tan et al. (64.3%-66.4%). Our model made more constraint violations compared to previous systems. This is due to the fact that our model uses independent classifiers to label each predicate-argument pair, making it difficult for them to implicitly track the decisions made for several arguments with the same predicate. Constrained decoding at test time is effective at eliminating all the core-role inconsistencies (shown in the U-column), but did not bring significant gain on the end result (shown in SRL F1), which only evaluates the piece-wise predicate-argument structures.