,
Table 26: Summary of the performances for different baselines. All numbers are in percentages.
['System', '[ITALIC] F1', '[ITALIC] EM']
The significant improvement brought by contextualized pre-training such as BERT and ELMo indicates that a significant portion of commonsense knowledge is actually acquired via pre-training. We can also see that human annotators achieved a very high performance under both metrics, indicating the high agreement level humans are on for this task. Our baselines, including BERT, still fall behind the human performance with a significantly margin.