Nurse is Closer to Woman than Surgeon? Mitigating Gender-Biased Proximities in Word Embeddings
Table 6: Comparison of various embedding methods for (a) analogy tests (performance is measured in accuracy) and (b) word semantic similarity tests (performance is measured in terms of Spearman rank correlation).
['[BOLD] Embedding', '[BOLD] (a) Analogy Google-Sem', '[BOLD] (a) Analogy Google-Syn', '[BOLD] (a) Analogy MSR', '[BOLD] (b) Semantic RG', '[BOLD] (b) Semantic MTurk', '[BOLD] (b) Semantic RW', '[BOLD] (b) Semantic MEN', '[BOLD] (b) Semantic SimLex999', '[BOLD] (b) Semantic AP']
The improvement in performance can be attributed to the removal of unwanted neighbours of a word vector (having gender bias based proximity), while enriching the neighbourhood with those having empirical utility, leading to a better performance in analogy tests. Word semantic similarity task is a measure of how closely a word embedding model captures the similarity between two words as compared to human-annotated ratings. For a word pair, we compute the cosine similarity between the word embeddings and its Spearman correlation with the human ratings. The results for these tests are obtained from the word embedding benchmark package Jastrzebski et al. Note that it is not our primary aim to achieve a state-of-the-art result in this test. It is only considered to evaluate semantic loss. This shows that RAN-Debias introduces minimal semantic disturbance.