KaLM at SemEval-2020 Task 4: Knowledge-aware Language Models for Comprehension And Generation
Table 2: Model performance on the official subtask B development set and test set. Extra Words: extra words indicating given statement does not make sense. Reasonable Statement: the corresponding reasonable statement. Wiktionary: the obtained evidence from wiktionary
['[EMPTY]', 'Dev Acc', 'Test Acc']
We see that the extra words can bring 1.6% absolute improvement since they indicate that the given statement does not make sense and the choice is the reason for that. We also see that using corresponding reasonable statement and wiktionary evidence achieve comparable performance while they involve extra computational cost. Therefore, we only use extra words in the final submission of subtask B and achieve accuracy of 93.2%