Racial Bias in Hate Speech and Abusive Language Detection Datasets
Table 4: Experiment 2, t= “b*tch”
['Dataset', 'Class', 'ˆ [ITALIC] piblack', 'ˆ [ITALIC] piwhite', '[ITALIC] t', '[ITALIC] p', 'ˆ [ITALIC] piblackˆ [ITALIC] piwhite']
The results for the second variation of Experiment 2 where we conditioned on the word "b*tch" are shown in Table 4. [CONTINUE] We see similar results for Waseem and Hovy (2016) and Waseem (2016). In both cases the classifiers trained upon their data are still more likely to flag black-aligned tweets as sexism. The Waseem and Hovy (2016) classifier is particularly sensitive to the word "b*tch" with 96% of black-aligned and 94% of white-aligned [CONTINUE] tweets predicted to belong to this class. For Davidson et al. (2017) almost all of these tweets are classified as offensive, however those in the blackaligned corpus are 1.15 times as frequently classified as hate speech. We see a very similar result for Golbeck et al. (2017) compared to the previous experiment, with black-aligned tweets flagged as harassment at 1.1 times the rate of those in the white-aligned corpus. Finally, for the Founta et al. (2018) classifier we see a substantial racial disparity, with black-aligned tweets classified as hate speech at 2.7 times the rate of white aligned ones, a higher rate than in Experiment 1.