Towards Robust and Privacy-preserving Text Representations
Table 1: POS prediction accuracy [%] using the Trustpilot test set, stratified by sex and age (higher is better), and the absolute difference (Δ) within each bias group (smaller is better). The best result is indicated in bold.
['[EMPTY]', 'sex F', 'sex M', 'sex Δ', 'age O45', 'age U35', 'age Δ']
Observe that the disparity for the baseline tagger accuracy (the Δ column), for age is larger than for sex, consistent with the results of \newcitehovy2015tagging. Our adv method leads to a sizeable reduction in the difference in accuracy across both sex and age, showing our model is capturing the bias signal less and more robust to the tagging task. Moreover, our method leads to a substantial improvement in accuracy across all the test cases. We speculate that this is a consequence of the regularising effect of the adversarial loss, leading to a better characterisation of the tagging problem.