Attending to Characters in Neural Sequence Labeling Models
Table 2: Comparison of word-based and character-based sequence labeling architectures on 8 datasets. The evaluation measure used for each dataset is specified in Section 6.
['[EMPTY]', 'BC2GM DEV', 'BC2GM TEST', 'CHEMDNER DEV', 'CHEMDNER TEST', 'JNLPBA DEV', 'JNLPBA TEST', 'GENIA-POS DEV', 'GENIA-POS TEST']
As can be seen, including a character-based component in the sequence labeling architecture improves performance on every benchmark. The NER datasets have the largest absolute improvement – the model is able to learn character-level patterns for names, and also improve the handling of any previously unseen tokens.