UM-IU@LING at SemEval-2019 Task 6: Identifying Offensive Tweets Using BERT and SVMs
Table 2: Results on the trial data for subtask A.
['[BOLD] System', '[BOLD] F1 macro', '[BOLD] Accuracy']
Generally, these variants of BERTbase tend to give similar performance but BERTbase-uncased achieved the best performance on the trial data. It is unclear why BERTbase-multilingual-uncased did not learn to perform the task beyond the baseline. Additional hyperparameter tuning might be necessary in this case. Overall, these results demonstrate that though BERT can give superior performance in detecting hate speech, it is somewhat sensitive to the change of hyperparameters. We also find that the SVM classifier achieved a higher accuracy on the trial data, but there is a significant drop in macro F1 when compared with the BERT model. This shows that the BERT model performs better on the minority class.