How Additional Knowledge can Improve Natural Language Commonsense Question Answering?
Table 1: Validation set accuracy (%) of each of the four models (Concat, Max, Simple sum, Weighted sum) across four datasets for each of the three strategies. The base model is BERT Large whole-word-masked.
['[BOLD] Dataset', '[BOLD] Strategy', '[BOLD] Concat ↑', '[BOLD] Max ↑', '[BOLD] Sim-Sum ↑', '[BOLD] Wtd-Sum ↑']
Which Strategy Works? We observe that knowledge helps in improving the performance. Both the Open Book and the Revision strategy perform well. Together the performance improves even further. The performance of the Revision strategy is poor for the SocialIQA and PFQA datasets. The reason behind this drop in performance may be due to the synthetic nature of the sentences and the unavailability of next sentence prediction task data. Note that the knowledge in the KB for SocialIQA and PFQA are single sentences and not paragraphs. The results for PIQA and aNLI datasets are better due to the presence of natural and contiguous knowledge sentences.