“Liar, Liar Pants on Fire”:A New Benchmark Dataset for Fake News Detection
Table 2: The evaluation results on the liar dataset. The top section: text-only models. The bottom: text + meta-data hybrid models.
['Models', 'Valid.', 'Test']
First, we compare various models using text features only. We see that the majority baseline on this dataset gives about 0.204 and 0.208 accuracy on the validation and test sets respectively. Standard text classifier such as SVMs and LR models obtained significant improvements. Due to overfitting, the Bi-LSTMs did not perform well. The CNNs outperformed all models, resulting in an accuracy of 0.270 on the heldout test set. We compare the predictions from the CNN model with SVMs via a two-tailed paired t-test, and CNN was significantly better (p