CJRC: A Reliable Human-Annotated Benchmark DataSet for Chinese Judicial Reading Comprehension
Table 6: Comparison data of unanswerable questions and “NO” questions, where unanswerable+ denotes adding extra unanswerable questions on the training set of the civil data.
['[EMPTY]', 'Number of Questions (Training set)', 'Number of Questions (Training set)', 'Number of Questions (Test set)', 'Number of Questions (Test set)', 'Performance (Test set)', 'Performance (Test set)']
It is easy to verify the first cause via observing the corpus. To verify the second point, we compare the unanswerable questions and the “NO” questions. The first two rows show that unanswerable questions presents a lower performance than the other on the criminal data, even though the former owns more questions. This has basically illustrated that the unanswerable questions are more hard. We have further experimented with increasing the number of unanswerable questions of civil data on the training set. However, despite having a larger amount of questions for unanswerable questions, it presents a lower score than “NO” questions.