FAKTA: An Automatic End-to-End Fact Checking System
Table 1: Results of document retrieval on FEVER.
['[BOLD] Model', '[BOLD] R@1', '[BOLD] R@5', '[BOLD] R@10', '[BOLD] R@20']
Document Retrieval: We use various search and ranking algorithms that measure the similarity between each input claim as query and Web documents. Lines 1–11 in the table show the results when we use Lucene to index and search the data corpus with the following retrieval models: BM25 Robertson et al. (Line 1), Classic based on the TF.IDF model (Line 2), and Divergence from Independence (DFI) Kocabaş et al. (Line 3). (Line 4) or a Zipfian relation prior (DFRz) (Line 5). We also consider Information Based (Line 6) or Smoothed power-law (IBSPL) (Line 7) distributions. According to the resulting performance at different ranks {1–20}, we select the ranking algorithm DFRz (LuceneDFRZ) as our retrieval model.