How Additional Knowledge can Improve Natural Language Commonsense Question Answering?
Table 2: Comparison of Performance of aNLI and SocialIQA datasets across multiple knowledge sources for Weighted-Sum model. For ANLI, Directly Derived refers to ROC-Stories and Story Cloze datasets. Related knowledge is Combined Commonsense Corpus. For SocialIQA, Partially Derived is the synthetic ATOMIC. For both, TrainOnly is the knowledge extracted only from the train set.
['[BOLD] Dataset', '[BOLD] RoBERTA+Knowledge Source', '[BOLD] Dev']
Which Strategy Works? We observe that knowledge helps in improving the performance. Both the Open Book and the Revision strategy perform well. Together the performance improves even further. The performance of the Revision strategy is poor for the SocialIQA and PFQA datasets. The reason behind this drop in performance may be due to the synthetic nature of the sentences and the unavailability of next sentence prediction task data. Note that the knowledge in the KB for SocialIQA and PFQA are single sentences and not paragraphs. The results for PIQA and aNLI datasets are better due to the presence of natural and contiguous knowledge sentences.