Real-time Claim Detection from News Articles and Retrieval of Semantically-Similar Factchecks
Table 3: Comparing Sentence Embeddings for Clustering News Claims.
['Embedding method', 'Time taken (s)', 'Number of claims', 'Number of clusters', 'Percentage of claims in', 'Percentage of claims in clusters']
This method is limited in how well it can represent the problem, but it can give indications as to a good or bad clustering method or embedding, and can act as a check that the findings we obtained from the Quora dataset will generalise to our domain. We used DBScan as it mirrored the clustering method used to derive the original article clusters. We included TFIDF in the experiment as a baseline to judge other results. That being said, TFIDF performs very well, with only Google USE Large and Infersent coming close in terms of ‘accuracy’. In the case of Infersent, this comes with the penalty of a much smaller number of claims included in the clusters. Google USE Large, however, clusters a greater number and for this reason we chose to use Google’s USE Large.