MAVEN: A Massive General Domain Event Detection Dataset
Table 4: The overall performances of different models on ACE 2005 and MAVEN. The DMCNN and MOGANED results on ACE 2005 are taken from original papers chen2015event; yan-etal-2019-event.
['Method', 'ACE 2005 P', 'ACE 2005 R', 'ACE 2005 F-1', 'MAVEN P', 'MAVEN R', 'MAVEN F-1']
It indicates that our MAVEN is challenging and the general domain ED still needs more research efforts. The deviations of DMBERT are larger than other models, which is consistent with recent findings dodge2020fine that the fine-tuning processes of pre-trained language models (PLM) are often unstable. It is necessary to run multiple times and report averages or medians for evaluating PLM-based models.