Self-Attention Gazetteer Embeddings for Named-Entity Recognition
Table 2: Results on CoNLL-03 and OntoNotes 5.
['[BOLD] Model', '[BOLD] Test F1±\xa0std CoNLL-03', '[BOLD] Test F1±\xa0std Ontonotes 5']
The top part of the table shows recently published results. Liu et al. ’s work is using gazetteers with HSCRF and Akbik et al. ’s work is using the Flair language model which is much larger than ELMo. Baevski et al. The bottom part of the table is shows our baseline models and results with included gazetteers. We experiment with the Neural CRF model with and without ELMo embeddings. Including ELMo embeddings the CoNLL-03 and Ontonotes 5, F1 score improves from 92.34 to 92.86 and 89.11 to 89.32 respectively. Without ELMo embeddings the F1 score improves from 90.42 to 91.12 and 86.63 to 87 respectively. We observe that GazSelfAttn relative improvements are similar with and without ELMo embeddings. We obtain slightly better CoNLL-03 F1 score compared to Liu et al. work that uses the HSCRF model, and we match the Ononotes 5 F1 scores of Akbik et al. that uses a much bigger model. Liu et al. Ononotes 5 results use subset of the dataset labels and are not comparable. Note that because of computation constrains, we did not perform extensive hyperparameter tuning except for the gazetteer dropout rate.