Events Beyond ACE: Curated Training for Events
Table 1: Teacher a vs NLP experts on conflict.demonstrate
['[EMPTY]', 'Tch. a', 'Designer', 'Exp. 1', 'Exp. 2']
The annotation tool designer and two other NLP experts also did ct for conflict.demonstrate All experts significantly outperformed Teacher a and ace in terms of F1. In two cases this is because the experts sacrificed precision for recall. The second expert matched Teacher a’s precision with much higher recall. Annotators varied widely in the volume of their annotation and indicator searches, but this did not have a clear relationship to performance.