Universal Sentence Encoder
Table 4: Word Embedding Association Tests (WEAT) for GloVe and the Universal Encoder. Effect size is reported as Cohen’s d over the mean cosine similarity scores across grouped attribute words. Statistical significance is reported for 1 tailed p-scores. The letters in the Ref column indicates the source of the IAT word lists: (a) greenwald1998 (b) bertrand2004 (c) nosek2002a (d) nosek2002b (e) monteith2011.
['Target words', 'Attrib. words', 'Ref', 'GloVe d', 'GloVe p', 'Uni. Enc. (DAN) d', 'Uni. Enc. (DAN) p']
\newcitecaliskan183’s findings on bias within GloVe embeddings with the DAN variant of the universal encoder. Similar to GloVe, our model reproduces human associations between flowers vs. insects and pleasantness vs. unpleasantness. However, our model demonstrates weaker associations than GloVe for probes targeted at revealing at ageism, racism and sexism. The differences in word association patterns can be attributed to differences in the training data composition and the mixture of tasks used to train the sentence embeddings.