Moving Down the Long Tail of Word Sense Disambiguationwith Gloss Informed Bi-encoders
Table 4: Performance of various pretrained encoders on the WSD development set.
['[BOLD] Pretrained Model', '[BOLD] Dev F1']
Next, we consider how the choice of pretrained model affects WSD performance. the WSD SemEval2007 evaluation set, which is used as our development set; we also consider the WSD performance of RoBERTa-base and RoBERTa-large Liu et al. Similarly to the pretrained BERT-base baseline from previous section, we do not fine-tune the pretrained encoders, as we found that for all considered pretrained encoders that this did not improve performance over the frozen model.