Identifying Purpose Behind Electoral Tweets
Table 13: Per category precision (P), recall (R), and F1 score of the classification on the 11-category problem. Micro-averaged P, R, and F1 are equal to accuracy since the categories are mutually exclusive.
['[BOLD] category', '[BOLD] # inst.', '[BOLD] P', '[BOLD] R', '[BOLD] F1']
As expected, the categories with larger amounts of labeled examples (‘to praise’, ‘to support’, ‘to provide information’) have higher results. However, for one of the higher frequency categories, ‘to ridicule’, the F1-score is relatively low. This category incorporates irony, sarcasm, and humour, the concepts that are hard to recognize, especially in a very restricted context of 140 characters. The four low-frequency categories (‘to agree’, ‘to point out mistake or blunder’, ‘to disagree’, ‘none of the above’) did not have enough training data for the classifier to build adequate models. The categories within ‘oppose’ are more difficult to distinguish among than the categories within ‘favour’. However, for the most part this can be explained by the larger number of categories (6 in ‘oppose’ vs. 3 in ‘favour’) and, consequently, smaller sizes of the individual categories.