Identity and Granularity of Events in Text
Table 8: Bag of events approach to event coreference resolution, evaluated on the ECB+ in MUC, B3, mention-based CEAF, BLANC and CoNLL F measures.
['Step1 Alg', 'Step1 Slot', 'Step1 Fea-', 'Step2 Alg', 'Step2 Slot', 'Step2 Fea-', 'MUC R', 'MUC P', 'MUC F', 'B3 R', 'B3 P', 'B3 F', 'CEAF F', 'BLANC R', 'BLANC P', 'BLANC F', 'CoNLL F']
To quantify the contribution of document features, we contrast the results of classifiers using bag of events features with scores achieved when disregarding document features. Note that one-step classification without preliminary document template clustering is computationally much more expensive than a two-step approach, which ultimately takes into account much less item pairs thanks to the initial document template clustering. The DT sentence template classifier trained on an unbalanced training set reaches 70% CoNLL F. This is 8% better than the strong baseline disregarding event arguments, but only 3% less than the two-step bag of events approach and 2% less than the one-step classification with document features. The reason for the relatively small contribution by document features could owe to the fact that in the ECB+ corpus not that many sentences are annotated per text. 1840 sentences are annotated in 982 corpus texts, i.e. 1.87 sentence per text. We expect that the impact of document features would be bigger, if more event descriptions from a discourse unit were taken into account than only the ground truth mentions.