Mind the GAP: A Balanced Corpus of Gendered Ambiguous Pronouns
Table 7: Performance of our baselines on the development set in the gold-two-mention task (access to the two candidate name spans). Parallelism+URL tests the page-context setting; all other test the snippet-context setting. Bold indicates best performance in each setting.
['[EMPTY]', 'M', 'F', 'B', 'O']
RANDOM is indeed closer here to the expected 50% and other baselines are closer to gender-parity. [CONTINUE] TOKEN DISTANCE and TOPICAL ENTITY are only weak improvements above RANDOM, [CONTINUE] Further, the cues are markedly gender-neutral, improving the Bias metric by 9% in the standard task formulation and to parity in the gold-two-mention case. [CONTINUE] .2% overall improvement over TRANSFORMER-SINGLE for the goldtwo-mention task.