PaRe: A Paper-Reviewer Matching Approach Using a Common Topic Space
Table 1: The mean precision of different baselines with optimal hyperparamters on the NIPS dataset. A reviewer is classified as relevant with a TREC score ≥2.
['[BOLD] Method', '[BOLD] Number of Topics', '[BOLD] P@5  [BOLD] GT1', '[BOLD] P@10  [BOLD] GT1', '[BOLD] P@5  [BOLD] GT2', '[BOLD] P@10  [BOLD] GT2', '[BOLD] P@5  [BOLD] GT3', '[BOLD] P@10  [BOLD] GT3']
Evaluation Metric. Precision at k (P@k) is a commonly-used ranking metric on NIPS dataset. P@k is defined to be the percentage of relevant reviewers in the top-k recommendations made by the model to a submission. It is likely that the top-k recommendations made by the model contain reviewers whose relevance information is not available in the ground truth. To address this issue, we first discard reviewers that do not have a relevance information prior to calculating P@k. In our experiments, we set k to be 5 and 10. We note that the results for APT 200 and Single Doc were only available for GT1 and we report them as such. Some approaches including Common Topic Model, Hidden Topic Model and LDA required a hyperparameter (number of topics) to be specified. As is shown, our proposed approach consistently outperforms the strong baselines. We also note that Hidden Topic Model and Doc2Vec are competitive approaches in expertise matching compared against probabilistic models.