CAiRE-COVID: A Question Answering and Multi-Document Summarization System for COVID-19 Research
Table A2: Results of our QA model on CovidQA dataset.
['Model', 'Precision', 'Recall@2']
The evaluation process of CovidQA dataset is designed as a supporting sentence selection task. Given one question and the corresponding articles, QA models are supposed to select and rank several sentences which possibly contains the golden answer from the whole article respectively. The results are evaluated using mean reciprocal rank (MRR), precision at rank one, and recall at rank three. In our system, the QA model performs in paragraph level, which make it hard to select several different possible sentences from the same context. In our case, we only use precision and recall for evaluation. For precision, we evaluation the ensemble result directly, while for recall@2, we separate the QA outputs from MRQA model and BioBERT model and consider the as two possible answers and compute the recall score.