MDR Cluster-Debias: A Nonlinear Word Embedding Debiasing Pipeline
Table 2: Performance on co-reference resolution task for models trained using the given embedding. All performance scores are given as F1 scores. Bolded results are the best-performing in each column.
['[BOLD] Model', '[BOLD] OntoNote', '[BOLD] Anti-Stereotype', '[BOLD] Pro-Stereotype', '[BOLD] WinoBias Mean', '[BOLD] WinoBias Diff.']
Among the embeddings considered, we find that MDR-Hard and MDR-Cluster show the best performance on the WinoBias datasets on WinoBias Difference and the Anti-Stereotype metric. This suggests that in addition to showing improvements on the tasks studied by Gonen and Goldberg, MDR-Hard and MDR-Cluster methods we propose can attenuate more protected gender information in the word embeddings. We further can find that different methods’ performances are similar on OntoNote dataset which indicates that our debias pipeline doesn’t deprecate the semantic information that are essential for coreference resolution. However, differences between the various debiasing strategies are limited, compared to their overall difference from the unbiased, original embeddings.