A Stylometric Inquiry into Hyperpartisan and Fake News
Table 4: Accuracies and class-wise precision, recall and F-Measure of predicting orientation.
['[BOLD] Features', '[BOLD] Accuracy all', '[BOLD] Precision left', '[BOLD] Precision right', '[BOLD] Precision main.', '[BOLD] Recall left', '[BOLD] Recall right', '[BOLD] Recall main.', '[BOLD] F [BOLD] 1 left', '[BOLD] F [BOLD] 1 right', '[BOLD] F [BOLD] 1 main.']
C. Predicting Orientation. As a last experiment in this series, we trained a classifier to predict the orientation of an individual news article. Interestingly, in the three-class setting, the topic baseline outperforms the style-based model with regard to accuracy, whereas the results for class-wise precision and recall are a mixed bag. The left-wing documents are apparently significantly more difficult to be identified compared to documents from the other two orientations. When looking at the confusion matrix, it turns out that 66% of misclassification of left-wing documents are falsely classified as right-wing documents. The reverse, however, is not true: 60% of all misclassified right-wing documents are classified as mainstream documents, whereas misclassified mainstream documents spread almost equally across the other classes.