Don’t Take the Easy Way Out:Ensemble Based Methods for Avoiding Known Dataset Biases
Table 10: Accuracy, and per-class scores, on the manually annotated questions for the various question classification methods we used when building TriviaQA-CP.
['Method', 'Accuracy', 'Location Precision', 'Location Recall', 'Location F1', 'Person Precision', 'Person Recall', 'Person F1']
The final model achieves about 95% accuracy. We find about 25% of the questions are about people and about 20% of the questions are about locations. Paragraph Selection: In TriviaQA, each question is paired with multiple documents. We simplify the task by selecting a single answer-containing paragraph for each question. We use the approach of clark2017simple to break up the documents into passages of at most 400 tokens, and rank the passages for each question using their linear paragraph ranker. Each question is then paired with the highest ranking paragraph that contains an answer.