End-to-End Open-Domain Question Answering with BERTserini
Table 1: Results on SQuAD development questions.
['Model', 'EM', 'F1', 'R']
We compare article retrieval at k=5, paragraph retrieval at k=29, and sentence retrieval at k=78. The article setting matches the retrieval condition in Chen et al. The values of k for the paragraph and sentence conditions are selected so that the reader considers approximately the same amount of text: each paragraph contains 2.7 sentences on average, and each article contains 5.8 paragraphs on average. The table also copies results from previous work for comparison. Our next experiment examined the effects of varying k, the number of text segments considered by the BERT reader. Here, we focus only on the paragraph condition, with μ=0.5 (the value learned via cross validation). Top k exact match represents a lenient condition where the system receives credit for a correctly-identified span in any retrieved segment. Scores for the paragraph condition at k=100 are also reported in the table: we note that the exact match score is substantially higher than the previously-published best result that we are aware of.