Towards Hate Speech Detection at Large via Deep Generative Modeling
Table 3: Intra-Dataset classification results, comparing the baseline and augmented-baseline training sets
['Dataset', 'Accuracy Baseline', 'Accuracy Augmented', 'Accuracy (%)', 'Precision Baseline', 'Precision Augmented', 'Precision (%)', 'Recall Baseline', 'Recall Augmented', 'Recall (%)', 'F1 Baseline', 'F1 Augmented', 'F1 (%)']
Intra-Dataset Experiments. The results indicate that augmenting the training sets with automatically generated examples leads to improvement in Recall and F1 in most cases, peaking at +73.0% and +33.1%, respectively, for the SE dataset. Precision on the other hand is decreased for most datsets, and Accuracy changes mildly. This is not surprising, as the training and test sets belong to the same distribution and are highly similar in this setup, whereas the generated text sequences introduce language diversity, as well as some label noise. For the combined dataset, which is relatively diverse, there is a yet a boost due to train set augmentation of +48.2% in Recall and +16.1% in F1; Precision decreases by 19.7% and Accuracy almost unchanged. Therefore, by dataset augmentation, a dramatic drop in false-negatives (missed hate speech sequences) is achieved, namely, significantly more hate speech is detected, which is also evident in the improved F1. The decrease in Precision indicates more false-positives (non-hate sequences classified as hate), however, these are less severe than missed hate speech sequences. These curves clearly demonstrates that training data augmentation yields significantly imrpved Recall (and consequently F1) across this range.