Fully Automated Fact Checking Using External Sources
Table 2: Results using an SVM with and without task-specific embeddings from the NN on the Rumor detection dataset. Training on Web pages vs. snippets vs. both.
['[BOLD] Model', '[BOLD] External support', '[BOLD] AvgR', '[BOLD] AvgF1', '[BOLD] Acc']
The NN embeddings consistently help the SVM in all cases. Moreover, while the baseline SVM using snippets is slightly better than when using Web pages, there is almost no difference between snippets vs. Web pages when NN embeddings are added to the basic SVM. Finally, gathering external support from either Google or Bing makes practically no difference, and using the results from both together does not yield much further improvement. Thus, (i) the search engines already do a good job at generating relevant snippets, and one does not need to go and download the full Web pages, and (ii) the choice of a given search engine is not an important factor. These are good news for the practicality of our approach.