Convolutional Neural Networks for Toxic Comment Classification
Table 1. Mean values and Standard Deviation across all experiments for Accuracy, Specificity and False discovery rate for all Classification Methods.
['[EMPTY]', 'Accuracy Mean', 'Accuracy Std', 'Specificity Mean', 'Specificity Std', 'False disc.rate Mean', 'False disc.rate Std']
A confusion matrix for each ’run’ was created by calculating values in main diagnostic tests such as, True positive rate (Recall), Positive predictive value (Precision), F1 score ( It is evident that both CNNrand and CNNfix models outperformed SVM, kNN, NB and LDA methods in all cases achieving accuracy almost over 90%. With respect to the toxic predictive value (Precision), CNN models had values above 90% while other methods range from 65 to 85 percent. Almost similar behavior is observed in the True Positive Rate (Recall) and in F1-score. NB and kNN methods had the worst results having the lowest values on precision and recall respectively. This means that kNN classifies several non-toxic comments as toxic and NB classifies several toxic comments as non-toxic. Also, CNN models has the lowest variance in all cases while CNNfix has the best performance with respect to precision and recall. Finally we observed that CNN models have the lowest False Discovery Ratio, meaning that they mistakenly predicted as ‘toxic’ the lowest number of ‘non-toxic’ comments. The aforementioned findings state clearly that CNN outperforms traditional text mining approaches for toxic comment classification presenting great potential for further development in toxic comment identification.