Comparing CNN and LSTM character-level embeddings in BiLSTM-CRF models for chemical and disease named entity recognition
Table 3: Results (in %) on the test set. [♠] denotes results reported on a 950/50 training/development split rather than our 900/100 split. As indicated, Att-BiLSTM-CRF used LSTM-char word embeddings.
['[BOLD] Model', '[BOLD] Chemical P', '[BOLD] Chemical R', '[BOLD] Chemical F1', '[BOLD] Disease P', '[BOLD] Disease R', '[BOLD] Disease F1', '[BOLD] Overall P', '[BOLD] Overall R', '[BOLD] Overall F1']
The first three rows show the performance of baseline models without the CRF layer, the next three rows show the performance of BiLSTM-CRF models without additional features, and then the next three rows show the results for BiLSTM-CRF models with additional gazetteer features. In particular, on chemical NER, both BiLSTM-CRF-based models with character-level word embeddings obtained the same F1 score (91.94%), while on disease NER the model with CNN-char obtained slightly higher performance (83.01%) than the model with LSTM-char (82.83%). All models with the CRF layer outperformed their respective baseline BiLSTM models in F1 scores for all entity categories. The pattern of chemical NER outperforming disease NER is consistent across all tools. our 900/100 split). Dnorm Leaman et al. The tmChem Leaman et al.