Commonsense Evidence Generation and Injection in Reading Comprehension
Table 1: Comparison of approaches on CosmosQA (Accuracy %) from the AI2 Leaderboard. T+F means using generated textual and factual evidence together.
['Model', 'Dev', 'Test']
Comparing to all methods, our proposed model CEGI(T+F) has the highest accuracy on both development set and test set. Most of the reading comprehension approaches utilize the attention mechanism to capture the correlations between paragraph, question and option set, therefore, the model tends to select the one option that is semantically closest to the paragraph. Among all of the group 1 methods, Multiway has the highest accuracy of 68.3%. Group 2 methods consider deep contextual representation of the given paragraph, question and option set, and increase the performance. Comparing group 2 methods with group 1 methods, RoBERTa-FT, which uses dynamic masking and large mini-batches strategy to train BERT, gains 11.1% accuracy increase compared to Multiway.