A Topic-Agnostic Approach for Identifying Fake News Pages
Table 3. Classification results (accuracies) for three datasets.
['[BOLD] Dataset  [BOLD] Features', '[BOLD] Celebrity  [BOLD] H', '[BOLD] Celebrity  [BOLD] C', '[BOLD] Celebrity  [BOLD] HC', '[BOLD] US-Election2016  [BOLD] H', '[BOLD] US-Election2016  [BOLD] C', '[BOLD] US-Election2016  [BOLD] HC', '[BOLD] PoliticalNews  [BOLD] H', '[BOLD] PoliticalNews  [BOLD] C', '[BOLD] PoliticalNews  [BOLD] HC']
Note that combining features often leads to the highest accuracies. For example, for political news, the highest accuracies are obtained by the configuration that combines LIWC (L) - NLTK (N) - readability (R) - webmarkup (W), which attains 0.86 accuracy for US-Election2016 and 0.83 for PoliticalNews. For the Celebrity data, the best configuration is LIWC (L) - readability (R) - webmarkup (W) which attains 0.78 accuracy. If we further examine the results from LIWC, we can see that combining LIWC with other features often leads to higher accuracies: for all datasets, the accuracy gains vary between 0.012 and 0.21. The web-markup features also lead to improvements, in particular when compared against readability for the PoliticalNews corpus. We also compare the best TAG feature set combination for each dataset, identified during our previous experiment, with the baseline. We use a linear SVM and five-fold cross-validation with accuracy as metric. Furthermore, it is important to note that FNDetector features uses 3 orders of magnitude more features (∼800,000) than our approach (∼160 features). The number of features has important implications for the processing time, maintaining the model, and explainability.