Talk to Papers: Bringing Neural Question Answering to Academic Search
Table 3: Main evaluation results.
['[EMPTY]', 'BM25 MRR', 'R@5', 'SOCO MRR', 'R@5']
The proposed SOCO-QA model is able to significantly outperform the baseline BM25 on all datasets. The proposed method is particularly powerful on real query data, e.g. NQ and MARCO where the question writer does not the exact answer they are looking for, so that there is often a low word overlapping between the question and the answer. On the other hand, SOCO is also able to beat BM25 on SQuAD and Trivia dataset, where there is significant more question-to-answer word lapping.