Using BERT for Word Sense Disambiguation
Table 4: Ablation Study. Mean and Max denotes the average and max operation for merging hidden states of polysemes. Concat means concatenating the hidden states of polysemes with the sentence vector.
['[EMPTY]', 'SE2', 'SE3', 'SE13', 'SE15', 'All']
Ablation Study We compare two variations of the BERT encoder: the way of merging hidden states of polysemes, and whether concatenating the hidden states of polysemes with the sentence vector. All of the models are based on Bertdef. We can find that the performance of using average or max operation is nearly the same for Bertdef. But Concatenating the sentence vector will impair the F1-score. The reason is: the sentence vector contains the global semantic information of the entire sentence. The sense of polyseme is determined by its local context. So using the sentence vector brings too much irrelevant information.