JEC-QA: A Legal-Domain Question Answering Dataset
Table 7: Evaluation results (accuracy %) of different models on JEC-QA. Results marked “-” indicates that the model cannot converge within 256 epochs.
['[EMPTY]', 'KD-questions Single', 'KD-questions All', 'CA-questions Single', 'CA-questions All', 'All Single', 'All']
We evaluate the performance of all models on JEC-QA, with settings of single-answer question and all question answering. Besides, we also evaluate the performance in KD-questions and CA-questions separately. In addition, we evaluate the performance of skilled and unskilled humans. Humans read the same paragraphs fetched by the search strategy as models do. Unskilled humans are those who do not have legal experience while skilled humans are those in legal professions.