Mind the GAP: A Balanced Corpus of Gendered Ambiguous Pronouns
Table 4: Performance of off-the-shelf resolvers on the GAP development set, split by Masculine and Feminine (Bias shows F/M), and Overall. Bold indicates best performance.
['[EMPTY]', 'M', 'F', 'B', 'O']
We note particularly the large difference in performance between genders, [CONTINUE] Both cues yield strong baselines comparable to the strongest OntoNotes-trained systems (cf. Table 4). In fact, Lee et al. (2017) and PARALLELISM produce remarkably similar output: of the 2000 example pairs in the development set, the two have completely opposing predictions (i.e. Name A vs. Name B) on only 325 examples.