FQuAD: French Question Answering Dataset
Table 7: Human Performance on FQuAD
['Dataset', 'F1 [%]', 'EM [%]']
The human score on FQuAD1.0 reaches 92.1% F1 and 78.4% EM on the test set and 92.6% and 79.5% on the development set. On FQuAD1.1, it reaches 91.2% F1 and 75.9% EM on the test set and 92.1% and 78.3% on the development set. We observe that there is a noticeable gap between the human performance on FQuAD1.0 test dataset and the human performance on the new samples of FQuAD1.1 with 78.4% EM score on the 2189 questions of FQuAD1.0 test set and 74.1% EM score on the 3405 new questions of FQuAD1.1 test set. This gap in human performance constitutes for us a proof that answering to FQuAD1.1 new questions is globally more difficult than answering to FQuAD1.0 questions, hence making the final FQuAD1.1 dataset even more challenging.