ARKref: a rule-based coreference resolution system
Table 2: Ablation analysis: Pairwise F1 performance on Bengston and Roth’s ACE dev set
['[BOLD] Main system', 'P 64.1', 'R 48.1', 'F1 55.0', '[EMPTY]']
This was performed with an earlier version of ARKref, thus the lower numbers for the “Main system.” However, the bulk of possible improvements to the system are still in other forms of pronoun and non-pronoun resolution. Another odd case is pronoun-to-pronoun matches, which we didn’t even consider when building the system. It turns out many of these work OK, even when matching between seemingly type-mismatches like “I” resolving to “he” — e.g. in dialogue or quotations.