Using BERT for Word Sense Disambiguation
Table 3: F1-score(%) on words with different occurrence numbers in the training dataset.
['Word Count', '0', '1-10', '11-50', '51-200', '>200']
Word Frequency We compare the performance of Bert and Bertdef on words with different occurrence numbers in the training dataset. Compared with the Bert model, Bertdef achieves the largest performance improvement on unseen words (8% F1-score). The improvement decreases as the word becomes frequent because the Bert model can be trained better with more instances. So we can conclude that utilizing sense definitions can enhance the model on infrequent polysemes. In addition, both models have better performance on low-frequency words. The reason is that high-frequency words usually have many senses.