Reading the Manual: Event Extraction as Definition Comprehension
Table 1: P(recision), R(ecall), and F1 obtained by models on the ACE 2005 dataset. Best results are bolded. Using the full training set improves F1 performance over using the partial training set on all metrics except argument identification (equal). Pre-training on the recast SQuAD 2.0 dataset improves F1 performance over no pre-training on all metrics.
['[BOLD] Model', '[BOLD] Trigger  [BOLD] Identification', '[BOLD] Trigger  [BOLD] Identification', '[BOLD] Trigger  [BOLD] Identification', '[BOLD] Trigger  [BOLD] Classification', '[BOLD] Trigger  [BOLD] Classification', '[BOLD] Trigger  [BOLD] Classification', '[BOLD] Argument  [BOLD] Identification', '[BOLD] Argument  [BOLD] Identification', '[BOLD] Argument  [BOLD] Identification', '[BOLD] Argument  [BOLD] Classification', '[BOLD] Argument  [BOLD] Classification', '[BOLD] Argument  [BOLD] Classification']
Training on the full training set improves F1 performance over training on the partial training set, giving the largest improvement on trigger identification and classification. Additionally, pre-training on the recast SQuAD 2.0 dataset provides large F1 improvements (4.8%–10.2% absolute F1 increase) on all four evaluation metrics. Our model also tends to have higher recall than precision, especially on trigger identification and classification, and suffers from low precision compared to prior work. Our model achieves state-of-the-art performance on trigger identification and trigger classification.