TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension
Table 7: Performance of all systems on TriviaQA using distantly supervised evaluation. The best performing system is indicated in bold.
['Method', 'Domain', 'Distant Supervision Dev', 'Distant Supervision Dev', 'Distant Supervision Dev', 'Distant Supervision Test', 'Distant Supervision Test', 'Distant Supervision Test', 'Verified Dev', 'Verified Dev', 'Verified Dev', 'Verified Test', 'Verified Test', 'Verified Test']
The poor performance of the random entity baseline shows that the task is not already solved by information retrieval. For both Wikipedia and web documents, BiDAF (40%) outperforms the classifier (23%). The oracle score is the upper bound on the exact match accuracy. All models lag significantly behind the human baseline of 79.7% on the Wikipedia domain, and 75.4% on the web domain.