To lemmatize or not to lemmatize: how word normalisation affects ELMo performance in word sense disambiguation
Table 6: F1 scores for the excluded target words from RUSSE’18.
['[BOLD] Word', '[BOLD] Tokens', '[BOLD] Lemmas', '[BOLD] STD']
(recall that they were excluded because of being ambiguous only in some inflectional forms). For these words we can see a great improvement with lemma-based models. This, of course stems from the fact that these words in different senses have different lemmas. Therefore, the results are heavily dependent on the quality of lemmatization.