Leveraging Affective Bidirectional Transformers for Offensive Language Detection
Table 1: Offensive (OFF) and Hate Speech (HS) Labels distribution in datasets
['[EMPTY]', 'Dataset', '[BOLD] #tweets', '[BOLD] # NOT_OFF', '[BOLD] # OFF', '[BOLD] OFF%', '[BOLD] # NOT_HS', '[BOLD] # HS', '[BOLD] HS %']
It is noteworthy that the dataset is imbalanced. For offensiveness (Sub_task_A), only 20% of the TRAIN split are labeled as offensive and the rest is not offensive. For hate speech (Sub_task_B), only 5% of the tweets are annotated as hateful. Due to this imbalanced, the official evaluation metric is macro F1 score. To add positive class data (i.e., ‘not-offensive’ and ‘not-hateful’) to this augmented collection, we randomly sample another 500K tweets that carry Ya from [abdul2019dianet] that do not carry any of the two offensive and hateful seed lexica. We apply AraNet on these tweets and keep only tweets assigned a positive sentiment label (%=70). We use 215,365 tweets as ‘non-offensive’ but only 199,291 as ‘non-hateful’.