SberQuAD – Russian Reading Comprehension Dataset: Description and Analysis
Table 8: Answer quality for misspelled questions.
['[EMPTY]', '[BOLD] % test', '[BOLD] R-Net', '[BOLD] BiDAF', '[BOLD] DocQA', '[BOLD] DrQA', '[BOLD] BERT']
DrQA seems to be most sensible to misspellings: The difference in F1 is almost 8% (scores are lower for misspelled questions). DocQA has most stable behavior: The difference in F1 scores is about 2%.