Towards Robust Toxic Content Classification
Table 4: Detailed results of models on two datasets. The best results for each setting are highlighted. C+D refers to character-level perturbations and distractors combined. BERT generally performs strongest in clean settings. The CDAE is better at handling noise without adversarial training, while BERT and CDAE perform comparably when adversarial training is introduced.
['Model', 'Train Noise', 'Test Noise', 'Jigsaw 2018 AUC', 'Jigsaw 2018 F1', 'Jigsaw 2018 Recall', 'OffensEval 2019 AUC', 'OffensEval 2019 F1', 'OffensEval 2019 Recall']
In order to evaluate our proposed defenses, we measure AUC, F1 score, and recall over the toxic class for all models. The model architecture for CDAE is similar to the one we used for fastText and ELMo. For CDAE, we use the mean of the final 4 layers of the model and concatenate them with fastText embeddings, because we found that this leads to superior performance. The detailed results of applying adversarial training as well as CDAE’s performance on the Jigsaw 2018 and OffensEval 2019