LogiQA: A Challenge Dataset for Machine Reading Comprehension with Logical Reasoning
Table 7: RoBERTa accuracy (%) against the reasoning type.
['[BOLD] Reasoning Type', '[BOLD] RoBERTa']
The method gives the best accuracy on categorical reasoning. However, for the other four reasoning types, the results are significantly lower. To understand why these tasks are challenging for RoBERTa, we give qualitative discussion via case study.