Racial Bias in Hate Speech and Abusive Language Detection Datasets
Table 3: Experiment 2, t= “n*gga”
['Dataset', 'Class', 'ˆ [ITALIC] piblack', 'ˆ [ITALIC] piwhite', '[ITALIC] t', '[ITALIC] p', 'ˆ [ITALIC] piblackˆ [ITALIC] piwhite']
Table 3 shows that for tweets containing the word "n*gga", classifiers trained on Waseem and Hovy (2016) and Waseem (2016) are both predict black-aligned tweets to be instances of sexism approximately 1.5 times as often as white-aligned tweets. The classifier trained on the Davidson et al. (2017) data is significantly less likely to classify black-aligned tweets as hate speech, although it is more likely to classify them as offensive. Golbeck et al. (2017) classifies black-aligned tweets as harassment at a higher rate for both groups than in the previous experiment, although the disparity is narrower. For the Founta et al. (2018) classifier we see that black-aligned tweets are slightly less frequently considered to be hate speech but are much more frequently classified as abusive.