Double-Hard Debias:Tailoring Word Embeddings for Gender Bias Mitigation
Table 6: WEAT test of embeddings before/after Debiasing. The bias is insignificant when p-value, p>0.05. Lower effective size (d) indicates less gender bias. Across all target words sets, Double-Hard Word2Vec leads to the smallest effective size. Specifically, for Science & Arts words, Double-Hard Word2Vec successfully reaches a bias insignificant state (p=0.09).
['[BOLD] Embeddings', '[BOLD] Career & Family  [ITALIC] d', '[BOLD] Career & Family  [ITALIC] p', '[BOLD] Math & Arts  [ITALIC] d', '[BOLD] Math & Arts  [ITALIC] p', '[BOLD] Science & Arts  [ITALIC] d', '[BOLD] Science & Arts  [ITALIC] p']
We further evaluate the debiasing outcome with WEAT test. Similar to experiments on GloVe embeddings, we use male names and female names as attribute sets and analyze the association between attribute sets and three target sets. Across three target sets, Double-Hard Word2Vec is able to consistently reduce the effect size. More importantly, the bias related to Science & Arts words becomes insignificant after applying our debiasing method.