Content-Based Citation Recommendation
Table 2: F1@20 and MRR results for two baselines and three variants of our method. BM25 results are based on our implementation of this baseline, while ClusCite results are based on the results reported in ren2014cluscite. “NNSelect” ranks candidates using cosine similarity between the query and candidate documents in the embedding space (phase 1). “NNSelect + NNRank” uses the discriminative reranking model to rerank candidates (phase 2), without encoding any of the metadata features. “+ metadata” encodes the metadata features (i.e., keyphrases, venues and authors), achieving the best results on all datasets. Mean and standard deviations are reported based on five trials.
['[BOLD] Method', '[BOLD] DBLP F1@20', '[BOLD] DBLP MRR', '[BOLD] PubMed F1@20', '[BOLD] PubMed MRR', '[BOLD] OpenCorpus F1@20', '[BOLD] OpenCorpus MRR']
Since the OpenCorpus dataset is much bigger, we were not able to train the ClusCite baseline for it. \newciteTotti2016AQA have also found it difficult to scale up ClusCite to larger datasets. Where available, we report the mean ± standard deviation based on five trials. Okapi-BM25 is a popular ranking function. We use BM25 as an IR-based baseline for the task of citation recommendation. For the DBLP and Pubmed datasets, BM25 performance is provided in Ren et al. To create a competitive BM25 baseline for OpenCorpus, we first created indexes for the DBLP and Pubmed datasets and tuned the query to approximate the performance reported in previous work. We extract the key terms (using Whoosh’s key_terms_from_text method) from the title and abstract of each query document. The key terms from the document are concatenated to form the query string.