On Measuring and Mitigating Biased Inferences of Word Embeddings
Table 15: The effect of attenuating gender bias on subword embeddings in BERT with the he-she direction and random vectors with difference (diff) from no attenuation.
['Gender (BERT)', 'Gender (BERT) NN', 'Gender (BERT) FN', 'Gender (BERT) T:0.5', 'Gender (BERT) T:0.7']
In each case we compare against projecting along a random direction (repeated 8 times) in place of the gender direction (from he-she). These each have negligible effect, so these improvements are significant. Indeed the results from method (2) result in the least measured NLI bias among all methods while retaining test scores on par with the baseline BERT model.