Weaver: Deep Co-Encoding of Questions and Documents for Machine Reading
Table 1: Number of questions for each dataset used in this paper. DS: distantly supervised training data. ∗: These training sets are not used as is because no paragraph is associated with each question. †: Corresponds to Wikihop and SQuAD development set. ‡: All figures for bAbI tasks refer to a single task.
['Dataset', 'Train Plain', 'Train DS', 'Test']
We test our model’s ability to answer questions with various types of context, synthetic stories, paragraphs, documents, and full encyclopedia (Wikipedia), using the datasets described in this section. Two evaluation metrics are used depending on the dataset: the exact string match (EM) between the predicted span and the answer string and the F1 score, which measures the harmonic mean of precision and recall at the token level. All scores reported in this paper are percentages.