Know What You Don’t Know: Unanswerable Questions for SQuAD
Table 5: Exact match (EM) and F1 scores between system predictions and plausible answers, in cases where the system made a false positive error.
['[EMPTY]', 'EM', 'F1']
For each system, we first isolated their false positive errors—cases where they predicted an answer to an unanswerable question—on the development set. Within this set of examples, we measured exact match and F1 scores between the system predictions and plausible answers. Plausible answers account for roughly half of the false positive errors made by each of the computer systems, as well as by human answerers. We conclude that the plausible answers in our dataset do indeed serve their purpose of being distracting spans that could be mistaken for the correct answer.