Improved Word Sense Disambiguation Using Pre-Trained Contextualized Word Representations
Table 2: English all-words task results in F1 measure (%), averaged over three runs. SemEval 2007 Task 17 (SE07) test set is used as the development set. We show the results of nearest neighbor matching (1nn) and linear projection, by simple last layer linear projection, layer weighting (LW), and gated linear units (GLU). Apart from BERT representation of one sentence (1sent), we also show BERT representation of one sentence plus one surrounding sentence to the left and one to the right (1sent+1sur). The best result in each dataset is shown in bold. Statistical significance tests by bootstrap resampling (∗: p<0.05) compare 1nn (1sent+1sur) with each of Simple (1sent+1sur), LW (1sent+1sur), GLU (1sent+1sur), and GLU+LW (1sent+1sur).
['[BOLD] System', '[BOLD] SE07', '[BOLD] SE2', '[BOLD] SE3', '[BOLD] SE13', '[BOLD] SE15', '[BOLD] Avg']
It is shown in the table that with the nearest neighbor matching model, BERT outperforms context2vec and ELMo. This shows the effectiveness of BERT’s pre-trained contextualized word representation. When we include surrounding sentences, one to the left and one to the right, we get improved F1 scores consistently.