Why Do Masked Neural Language Models Still Need Common Sense Knowledge?
Table 5: Empirical analysis on the performances when adapting an external common sense repository. In the table, C2T is an abbreviation of ‘common sense to text’ indicating that we integrate the external common sense repository to the MNLMs.
['Model', 'has answer f1', 'has answer exact', 'no answer accuracy', 'overall f1', 'overall exact']
The results present that integrating the external common sense repository improves the performance of MNLMs. Especially, all questions except for one question are common sense knowledge or synonymy types. It implies that coping with external common sense knowledge can be a solution for complementing the weakness of MNLMs.