We Built a Fake News & Click-bait Filter:What Happened Next Will Blow Your Mind!
Table 3: Performance of different models.
['[BOLD] Feature Group', '[BOLD] P', '[BOLD] R', '[BOLD] F1', '[BOLD] Acc']
Evaluating the final model, we set as a baseline the prediction of the majority class, i.e., the fake news class. This baseline has an F1 of 41.59% and accuracy of 71.22%. Another stable baseline, apart from just taking the majority class, is the TF.IDF bag-of-words approach, which sets a high bar for the general model score. We then observe how much the attention mechanism embeddings improve the score (AttNN). Finally, we add the hand-crafted features (Feats), which further improve the performance. From the results, we can conclude that both the attention-based task-specific embeddings and the manual features are important for the task of finding fake news.