Contextualized Sparse Representations forReal-Time Open-Domain Question Answering
Table 1: Results on two open-domain QA datasets. See Appendix A for how s/Q is computed.
['[BOLD] Model', 'C.TREC EM', 'SQuAD-Open EM', 'SQuAD-Open F1', 's/Q']
On both datasets, our model with contextualized sparse representations ( DenSPI + Sparc) largely improves the performance of the phrase-indexing baseline model (DenSPI) by more than 4%. Also, our method runs significantly faster than other models that need to run heavy QA models during the inference. On CuratedTREC, which is constructed from real user queries, our model achieves state-of-the-art performance at the time of submission. Even though our model is only trained on SQuAD (i.e., zero-shot), it outperforms all other models which are either distant- or semi-supervised with at least 45x faster inference.