Event Extraction by Answering (Almost) Natural Questions
Table 4: Evaluation on unseen argument roles.
['[EMPTY]', 'Argument ID + Classification P', 'Argument ID + Classification R', 'Argument ID + Classification F1']
, we conduct another experiment, where we keep 80% of the argument roles (16 roles) seen at training time, and 20% (6 roles) only seen at test time. Specifically, the unseen roles are “Vehicle, Artifact, Target, Victim, Recipient, Buyer”. Random NE is our random baseline that selects a named entity in the sentence, it comes with a reasonable performance of near 25%. Prior models such as GAIL is not capable of handling the unseen roles. Using our QA-based framework, as we leverage more semantic information and naturalness into the question (from question template 1 to 2, to 3), both the precision and recall increases substantially.