On Measuring and Mitigating Biased Inferences of Word Embeddings
Table 7: Fraction of the top principal value with the xth principal value with the GloVe embedding for Gendered, Demonym, and Adherent datasets. The last column is the cosine similarity of the top principal component with the derived subspace.
['Embedding', '2nd', '3rd', '4th', 'cosine']
For nationalities, we identify a separate set of 8 demonyms than those used to create sentence pairs as DemonymTrain, and use their first principal component to define a 1-dimensional demonym subspace. For religions, we similarly use a AdherentTrain set, again of size 8, but use the first 2 principal components to define a 2-dimensional religion subspace. In both cases, these were randomly divided from full sets Demonym and Adherent. Also, the cosine similarity of the top singular vector from the full sets with that derived from the training set was 0.56 and 0.72 for demonyms and adherents, respectively. Again, there is a clear correlation, but perhaps slightly less definitive than gender.