Structured Tuning for Semantic Role Labeling
Table 7: Label-wise F1 scores for the CoNLL-05 and CoNLL-12 development sets.
['[EMPTY]', 'CoNLL-05 3% RoBERTa2', 'CoNLL-05 3% +U,F,O', 'CoNLL-05 100% RoBERTa2', 'CoNLL-05 100% +U,F,O', 'CoNLL-12 3% RoBERTa2', 'CoNLL-12 3% +U,F,O', 'CoNLL-12 100% RoBERTa2', 'CoNLL-12 100% +U,F,O']
Under low training data conditions, our constrained models gained improvements primarily from the frequent labels, \eg, A0-A2. On CoNLL-05 dataset, we found the location modifier (AM-LOC) posed challenges to our constrained models which significantly performed worse than the baseline. Another challenge is the negation modifier (AM-NEG), where our models underperformed on both datasets, particularly with small training data. When using the CoNLL-12 training set, our models performed on par with the baseline even on frequent labels, confirming that the performance of soft-structured learning is nearly saturated on the larger, cleaner dataset.