Automatic Detection of Fake News
Table 6: Cross-domain classification accuracy for the complete LIWC and readability feature sets
['Domain', 'Readability Acc.', 'Readability F1 [ITALIC] Legitimate', 'Readability F1 [ITALIC] Fake', 'Complete LIWC Acc.', 'Complete LIWC F1 [ITALIC] Legitimate', 'Complete LIWC F1 [ITALIC] Fake', 'All features Acc.', 'All features F1 [ITALIC] Legitimate', 'All features F1 [ITALIC] Fake']
We also assess the cross-domain classification performance for the six news domains in the FakeNewsAMT dataset. We do this by training on five of the six domains in the datset, and testing the remaining one. The politics, education, and technology domains appear to be rather robust against classifiers trained on other domains. The technology and politics domains, moreover, are classified both with a high accuracy of 0.91 with the Readability feature set, which may suggest that fake and legitimate news in each of these three domains might be structurally similar to the fake and legitimate content in the other five domains. By contrast, domains such as sports, business and entertainment are less generalizable and might therefore be more domain-dependent. Although further research is needed to consolidate these findings, a possible explanation could be the rather unique content and style of these domains