Answering Questions on COVID-19 in Real-Time
Table 4: Results on COVID-19 Questions
['[BOLD] Model', '[BOLD] Articles', 'Train', 'Features', 'Interrogative EMsent@1', 'Interrogative EMsent@50', 'Keyword EMsent@1', 'Keyword EMsent@50', 's/Q']
When DenSPI + Sparc is trained on SQuAD, its performance on interrogative questions is generally superior to other models. Although the performance on keyword questions when trained on both SQuAD and NQ improves with 2020-04-10 articles, it does not perform any better with 2020-04-10-recent articles. Lastly, we include our recent ongoing effort to improve the performance of DenSPI (DenSPI (unpublished)) which is purely trained on Natural Questions. Its performance on keyword questions largely outperforms other models. As shown in the last row, we continue to update covidAsk with updated versions of CORD-19 (2020-06-14-recent). Answerability of questions for a large amount of unstructured text: Although we tried to set a threshold for the scores of answers, the distribution of scores were very inconsistent across different questions (e.g., a top 1 answer with a low score is often correct). Leveraging domain adapted datasets: We tested whether using the BioASQ dataset