A Stylometric Inquiry into Hyperpartisan and Fake News
Table 2: Accuracies of predicting hyperpartisanship vs mainstream under different training sets.
['[BOLD] Features', '[BOLD] Training set  [BOLD] without left', '[BOLD] Training set  [BOLD] without left', '[BOLD] Training set  [BOLD] without left', '[BOLD] Training set  [BOLD] without right', '[BOLD] Training set  [BOLD] without right', '[BOLD] Training set  [BOLD] without right', '[BOLD] Training set  [BOLD] with both', '[BOLD] Training set  [BOLD] with both', '[BOLD] Training set  [BOLD] with both']
Besides our style feature model, we also provide performance values of corresponding classifiers trained under a topic feature model (bag of words) as baseline for comparison. When omitting left-wing documents from the training set, the style-based classifier still achieves 0.74 accuracy on left-wing test documents. Likewise, it achieves 0.66 accuracy classifying right-wing documents without having trained on them. In both cases, the style-based classifier outperforms the baseline topic feature model. This behavior supports our observation that the style of left-wing documents is not so different from that of right-wing ones. Unsurprisingly, not omitting any documents from the training data improves accuracy significantly, however, only for the class of hyperpartisan documents. The mainstream class can be best discriminated when omitting right-wing documents, at the expense of accuracy for the class of hyperpartisan documents.