JEC-QA: A Legal-Domain Question Answering Dataset
Table 8: Performance of Co-matching on different questions.
['[EMPTY]', 'KD-Q', 'CA-Q', 'All']
From the results we can see that existing methods can only answer about 32% of questions correctly even when there is enough evidence in reading paragraphs, which means that the models cannot understand the reading materials at all. Moreover, we can see that the model performs extremely bad on multi-paragraph reading and multi-hop reasoning questions of CA-questions. It means existing models cannot do multi-paragraph reading and multi-hop reasoning on real cases properly.