A Stylometric Inquiry into Hyperpartisan and Fake News
Table 6: Accuracies, class-wise precision, recall, and F-Measure of predicting satire. Also shows the results for the best classifier of rubin:2016.
['[BOLD] Features', '[BOLD] Accuracy all', '[BOLD] Precision sat.', '[BOLD] Precision real', '[BOLD] Recall sat.', '[BOLD] Recall real', '[BOLD] F [BOLD] 1 sat.', '[BOLD] F [BOLD] 1 real']
C. Discriminating satire. Finally, we evaluate the use of style features for predicting satire. This setting uses a balanced 3:1 training-to-test set split over 360 articles (180 per class). As can be seen, our style-based model significantly outperforms all baselines across the board, achieving an accuracy of 0.82, and an F score of 0.81. While it does not outperform the classifier based on topic, absurdity, grammar, and punctuation features by \newciterubin:2016, it clearly improves over a pure topic classification. We argue that incorporating topic into satire detection is not appropriate, since the topics of satire change as the topics of news in general do. A classifier with topic features would therefore not generalize. Apparently, a style-based model is competitive, and we believe that satire can be detected at scale this way, so as to prevent other fake news detection technology from falsely filtering it.