Birds have four legs?! NumerSense: Probing Numerical Commonsense Knowledgeof Pre-trained Language Models
Table 2: Results (%) of PTLMs on NumerSense. ‘Ft.’ stands for ‘Fine-tuned.’ The human performance is shown by closed testing (α=‘no external information’) / open testing (β=‘Wikipedia is allowed’).
['[BOLD] Models', '[BOLD] Core  [BOLD] hit@1', '[BOLD] Core hit@2', '[BOLD] Core hit@3', '[BOLD] Core + Adversarial  [BOLD] hit@1', '[BOLD] Core + Adversarial hit@2', '[BOLD] Core + Adversarial hit@3']
The first four lines are results from PTLMs in the zero-shot inference setting. We see that size matters, as there is a clear performance gain when the model sizes increases. Also, RoBERTa’s results are consistently better than BERT’s (37.63 vs 41.78), which is probably because RoBERTa uses a larger training corpora and focuses more on masked language modeling in its pre-training stage.