Nurse is Closer to Woman than Surgeon? Mitigating Gender-Biased Proximities in Word Embeddings
Table 7: F1-Score (in %) in the task of coreference resolution. Diff denotes the absolute difference between F1-score on PRO and ANTI datasets.
['[BOLD] Embedding', '[BOLD] OntoNotes', '[BOLD] PRO', '[BOLD] ANTI', '[ITALIC]  [BOLD] Diff']
Following the coreference resolution testing methodology used by Zhao et al. set for Type 1 template, along with the absolute difference (Diff) of F1-scores on PRO and ANTI datasets for different word embeddings. The results for GloVe, Hard-GloVe, and GN-GloVe are obtained from Zhao et al. This lower Diff is achieved by an improved accuracy in ANTI and a reduced accuracy in PRO. We hypothesise that the high performance of non-debiased GloVe in PRO is due to the unwanted gender cues rather than the desired coreference resolving ability of the model. Further, the performance reduction in PRO for the other debiased versions of GloVe also corroborates this hypothesis. Despite debiasing GloVe, a considerable amount of gender cues remain in the baseline models as quantified by a lower, yet significant Diff. In contrast, RAN-GloVe is able to remove gender cues dramatically, thereby achieving an almost ideal Diff. Additionally, the performance of RAN-GloVe on the OntoNotes 5.0 test set is comparable with that of other embeddings.