Multi-Task Learning with Contextualized Word Representations for Extented Named Entity Recognition
Table 4: 5 most improved NE types when using ELMo.
['Named Entity', 'GloVe', 'ELMo', 'Token Length']
While the average token length of NEs in our dataset is 1.9, the average token lengths of these NE types are much longer. It shows that ELMo helps to improve the performance of our system when identifying NEs which are long sequences. This result is understandable because Base Model (GloVe) relies on only BLSTM layer to learn the dependencies among words in sequence to predict NE labels while Base Model (ELMo) learns these dependencies by both embedding and BLSTM layers. Unlike NER, NE types in FG-NER are often more complex and longer so using only BLSTM layer is not sufficient to capture these dependencies.