Towards Hate Speech Detection at Large via Deep Generative Modeling
Table 4: Cross-Dataset performance, comparing detector training using baseline to augmented-baseline training sets
['Trainset- Testset', 'Accuracy Baseline', 'Accuracy Augmented', 'Accuracy (%)', 'Precision Baseline', 'Precision Augmented', 'Precision (%)', 'Recall Baseline', 'Recall Augmented', 'Recall (%)', 'F1 Baseline', 'F1 Augmented', 'F1 (%)']
In another set of experiments, we re-trained the detector with each one of the corresponding augmented training set, and evaluated on the same cross-dataset combinations of the first experiment. The average metrics of all cross-dataset pairs reveals a consistent increase in accuracy (+6.9%), precision (+3.5%), recall (+182.8%) and F1 (+179.7%). That is, augmenting the original labeled training sets with the automatically generated text sequences resulted in a dramatic increase of recall, while maintaining good levels of precision. Overall, this trend is reflected by a steep increase in the combined measure of F1. These results thus provide another approval of the benefits of augmenting the data with generated sentences and using a broader data distribution by utilizing a pre-trained language model.