Understanding Undesirable Word Embedding Associations
Table 1: By contriving the male and female attribute words, we can easily manipulate WEAT to claim that a given target word is more female-biased or male-biased than another. For example, in the top row, →door is more male-associated than →curtain when the attribute words are ‘masculine’ and ‘feminine’, but it is more female-associated when the attribute words are ‘woman’ and ‘man’. In both cases, the associations are highly statistically significant.
['Target Word Sets', 'Attribute Word Sets', 'Test Statistic', 'Effect Size', '[ITALIC] p-value', 'Outcome (WEAT)']
This also means that we can contrive the attribute word sets to achieve a desired outcome. For example, when the attribute word sets are {‘masculine’} and {‘feminine’}, →door is significantly more male-associated than →curtain. When the attribute sets are {‘woman’} and {‘man’}, the opposite is true: →door is significantly more female-associated than →curtain. Conversely, we can also manipulate the attribute sets to claim that an association is not statistically significant (p=0.5), despite a large effect size.