Towards Robust Toxic Content Classification
Table 6: Results of the Ensemble of CDAE and BERT. The ensemble performs the strongest against combined noise among all of our methods.
['Model', 'Train Noise', 'Test Noise', 'Jigsaw 2018 AUC', 'Jigsaw 2018 F1', 'Jigsaw 2018 Recall', 'OffensEval 2019 AUC', 'OffensEval 2019 F1', 'OffensEval 2019 Recall']
Ensembling. Based on our findings, we also examine the performance of an ensemble of BERT and the CDAE, in the hope that it will combine their advantages. The final prediction is made with arithmetic mean of the two models’ predicted probabilities. Indeed, the ensemble outperforms both the single CDAE and BERT models when tested on combined noise, exactly because it combines their different robustness characteristics. This suggests that although it may be difficult to train a single model to be robust to all possible attacks, specialized models can be trained to handle different attacks and their ensemble may be a simple, cheap approach that will boost robustness of the entire system.