Neutralizing Gender Bias in Word Embedding with Latent Disentanglement and Counterfactual Generation
Table 1: Percentage of predictions for each category on gender relational analogy task. We can expect a high percentage for Definition and low percentages for Stereotype and None for well-debiased word embeddings. † and ∗ denote the statistically significant differences comparing with Hard-GloVe and Glove, respectively. The best performing model is indicated as boldface.
['Embeddings', 'Sembias Definition ↑', 'Sembias Stereotype ↓', 'Sembias None ↓', 'Sembias subset Definition ↑', 'Sembias subset Stereotype ↓', 'Sembias subset None ↓']
4.3.1 Sembias Analogy Test We perform the gender relational analogy test with the Sembias dataset Zhao et al. ; Jurgens et al. The dataset contains 440 instances, and each instance consists of four pairs of words: 1) a gender-definition word pair (Definition), 2) a gender-stereotype word pair (Stereotype), and 3,4) two none-type word pairs (None). A tested model chooses a word pair (a,b) whose difference vector, →a−→b, has the highest cosine similarity with −→he−−−→she as a classification for the gender-definition word pair. By following the past practice Zhao et al. whose gender-definition word pairs are not used for training. Our model clearly selects all the gender-definition word pairs, which demonstrates the maintenance of the gender latent information for those words. Also, our model selects neither gender-stereotype words nor none-type words, so the difference vector of →a−→b has a minimal linear correlation with those words after applying our debiasing method.