Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing
Table 11. Comparison of entity-level F1 for biomedical named entity recognition (NER) using different tagging schemes and the standard PubMedBERT.
['1pt1.5pt1.5pt Tagging Scheme', 'BIO', 'BIOUL', 'IO']
We also investigate the tagging scheme used in NER. The standard tagging scheme distinguishes words by their positions within an entity. For sequential tagging methods such as CRF and LSTM, distinguishing the position within an entity is potentially advantageous compared to the minimal IO scheme that only distinguishes between inside and outside of entities. But for BERT models, once again, the utility of more complex tagging schemes is diminished. We thus conducted a head-to-head comparison of the tagging schemes using three biomedical NER tasks in BLURB.