Understanding and Detecting Dangerous Speech in Social Media
Table 8: Results from our models on TEST.
['[BOLD] Models', '[BOLD] Datasets', '[BOLD] Precision', '[BOLD] Recall', '[BOLD] Acc', '[BOLD] F1']
Offensive speech data. In one of our settings, we also use the offensive dataset released via the Offensive Shared Task 2020. This offensive content dataset consists of 8000 tweets (1,590 ‘offensive’ and 6,410 ‘non-offensive’). We use the offensive class data to augment our train split. Hence, we evaluate only on our test split where tweets are restricted to our dangerous gold tweets in the annotated dataset. We run this experiment as a way to test the utility of exploiting offensive tweets for enhancing dangerous language representation based on the assumption that dangerous speech is a subset of offensive language. In fact, it leads to model deterioration. Results & Discussion. We also observe that training on the offensive dataset did not improve the results. On the contrary, augmenting training data with the offensive task tweets cause deterioration to 53.52% F1 for BERT and 54.11% F1 for BERT-Emotion.