A Dictionary-based Approach to Racism Detection in Dutch Social Media
Table 6: P, R, F and ROC-AUC scores on the test set.
['[EMPTY]', 'P', 'R', 'F', 'AUC']
Our best-performing model was based on the expanded and cleaned version of the dictionary, but this model only slightly outperformed the other models. Additionally, we also computed Area Under the Receiving Operator Characteristic Curve (ROC-AUC) This shows that all dictionaries have comparable AUC scores, and that each dictionary outperforms the unigram baseline. To obtain additional evidence, we computed the statistical significance of performance differences between the models based on the dictionaries and unigram baseline model using approximate randomization testing (ART) An ART test between dictionary models reveals that none of the models had performance differences that were statistically significant. Similarly, all dictionary models outperformed the unigram baseline with statistical significance, with p < 0.01 for the models based on the cleaned and expanded dictionaries, and p < 0.05 for the models based on the original dictionary.