Monitoring stance towards vaccination in Twitter messages
Table 11: Performance of the ensemble system on correctly predicting tweets labeled as ‘Negative’ (AUC = Area under the curve, Total = all tweets in the test set that are labeled as ‘Negative’, Predicted = the number of tweets that were classified as ‘Negative’ by the system, Correct = the number of tweets that were correctly classified as ‘Negative’.
['Precision', 'Recall', 'F1-score', 'AUC', 'Total', 'Predicted', 'Correct']
Of the 343 tweets in the test set that are labeled as Negative, 210 are retrieved by the ensemble system. The result is a recall of 0.61. The system does overshoot in its categorization of tweets as Negative: this category is predicted for 1,168 tweets (about 40% of total test set of 2,886 tweets). The result is a precision of 0.18. In comparison to lowering the prediction threshold of the ML system, the ensemble system thus yields a slightly worse trade-off between precision and recall.