Improving Lemmatization of Non-Standard Languages with Joint Learning
Table 2: Accuracy for the Gysseling subcorpora.
['[EMPTY]', 'Full', 'Ambiguous', 'Unknown']
The model described by Kestemont et al. It is apparent that both Sent and Sent-LM outperform K-2016 on full and unknown tokens. It is worth noting that K-2016, a model that uses distributed contextual features but no edit-tree induction, performs better than Plain -- which highlights the importance of context for the lemmatization of historical languages --, and also better than the edit-tree approaches -- which highlights the difficulty of tree induction on this dataset. We find Sent-LM to have a significant advantage over Sent on full and ambiguous tokens, but a disadvantage vs Sent and Plain on unknown tokens.