Answering Questions on COVID-19 in Real-Time
Table 5: Results on TREC-COVID Round 1. For ‘Manual’ submission, we manually chose better results from dmis-rnd1-run1 and dmis-rnd1-run2 for each query.
['[BOLD] Team', '[BOLD] Run', 'Train', 'Features', 'P@5', 'NDCG@10', 'MAP', 'Bpref']
The results of our submissions (KoreaUniversity_DMIS) show that covidAsk is capable of performing IR, although it is not very effective compared to other systems that are fully dedicated to IR rather than QA. This is due to the fact that QA and IR are fundamentally different tasks, as it is also possible for documents unrelated to the question itself to contain correct answers. The discrepancy between evaluation metrics used for QA and IR also imply that the two tasks put emphasis on different aspects. For more details regarding TREC-COVID Round 1 submissions, please refer to the TREC-COVID Round 1 Archive: https://ir.nist.gov/covidSubmit/archive/archive-round1.html.