Multi-Task Learning with Contextualized Word Representations for Extented Named Entity Recognition
Table 3: Results in F1 scores for FG-NER (We run each setting five times and report the average F1 scores.)
['Model', 'FG-NER', '+Chunk', '+NER (CoNLL)', '+POS', '+NER (Ontonotes)']
Deep Contextualized Word Representations In the first experiment, we investigate the effectiveness of contextualized word representations (ELMo) compared to uncontextualized word representations (GloVe) when incorporating in our FG-NER systems (Base Model (GloVe) vs. Base Model (ELMo)). In both cases, hierarchical-shared model gives the best performances. In particular, it achieves an F1 score of 82.14% when learning with NER (CoNLL) and an F1 score of 83.04% when learning with NER (Ontonotes) compared to F1 scores of 81.51% and 82.74% of base model in GloVe and ELMo settings respectively. For same-level-shared models, they also achieve better results compared to base model but the differences are not very large. These results indicate that learning FG-NER with other sequence labeling tasks at different parameter sharing schemes helps to improve the performances of FG-NER system. Also, in most cases, it is more beneficial when learning the auxiliary and the main tasks at different levels (hierarchical-shared model) compared to learning at the same level (RNN-shared and embedding-shared models).