Neutralizing Gender Bias in Word Embedding with Latent Disentanglement and Counterfactual Generation
Table 2: WEAT hypothesis test results for five popular gender-biased word categories. The best performing model is indicated as boldface. The second-best model is indicated as underline. The absolute value of the effect size denotes the degree of bias, and the p-value denotes the statistical significance of the results.
['Embeddings', 'B1 : career vs family p-value', 'B1 : career vs family Effect size', 'B2 : maths vs arts p-value', 'B2 : maths vs arts Effect size', 'B3 : science vs arts p-value', 'B3 : science vs arts Effect size', 'B4 : intelligence vs appearance p-value', 'B4 : intelligence vs appearance Effect size', 'B5 : strength vs weakness p-value', 'B5 : strength vs weakness Effect size']
To quantify the degree of gender bias, we apply the Word Embedding Association Test (WEAT) Caliskan et al. WEAT measures the effect size and the hypothesis statistics based on the gender-definition words and the well-known gender-stereotypical words set, such as strength and weakness. While all baseline models record worse performance than GloVe on at least one of the categories, our model always shows better performances than GloVe.