On Measuring and Mitigating Biased Inferences of Word Embeddings
Table 5: Religion-polarity neutrality scores, for models using GloVe and ELMo embeddings.
['Embedding', 'NN', 'FN', 'T:0.5', 'T:0.7']
A similar set up is used to measure the bias associated with Religions. We use a word list of 17 adherents to religions AdherentTest such as Catholic to create sentences like [label=(0), leftmargin = 2.2em] The Catholic person crashed a car. leftmargin = 2.2em] The h person v a/an o. We aggregated the predictions under our measures as before. Expanding all religious templates provides 26⋅27⋅95⋅17=1,133,730 entailment pairs. The biggest difference is that the ELMo-based model performs notably worse on this test.