KaLM at SemEval-2020 Task 4: Knowledge-aware Language Models for Comprehension And Generation
Table 1: Model performance on the official subtask A development set and test set. Lowercase: make all the letters in lowercase except for the first letter. Evidence: add evidence from wiktionary
['[EMPTY]', 'Dev Acc', 'Test Acc']
We notice that there are some statements whose letters are all capitalized (e.g., A GIRL WON THE RACE WITH HER FRIEND) in Subtask A dataset. We capitalize the first letter and make other letters in lowercase. Then we explore the effect of evidence. By adding evidence to the input statement, we obtain a slightly better result in the development dataset while performance degradation is found in the test dataset. We hypothesize this discrepancy is because the amount of noise data in evidence is unstable.