Graph-Based Decoding for Event Sequencing and Coreference Resolution
Table 2: Test Results for Event Coreference with the Singleton and Matching baselines.
['[EMPTY]', '[ITALIC] B3', 'CEAF-E', 'MUC', 'BLANC', 'AVG.']
Comparing to the top 3 coreference systems in TAC-KBP 2015, we outperform the best system by about 2 points absolute F-score on average. Our system is also competitive on individual metrics. Our model performs the best based on B3 and CEAF-E, and is comparable to the top performing systems on MUC and BLANC. Note that while the Matching baseline only links event mentions based on event type and realis status, it is very competitive and performs close to the top systems. This is not surprising since these two attributes are based on the gold standard. To take a closer look, we conduct an ablation study by removing the simple match features one by one. We observe that some features produce mixed results on different metrics: they provide improvements on some metrics but not all. This is partially caused by the different characteristics of different metrics. On the other hand, these features (parsing and frames) are automatically predicted, which make them less stable. Furthermore, the Frame features contain duplicate information to event types, which makes it less useful in this setting.