Enhancing Clinical Concept Extraction with Contextual Embeddings
Table 5: Performance of each label category with pre-trained MIMIC models on i2b2 2012 task.
['2.5', '[BOLD] word2vec', '[BOLD] GloVe', '[BOLD] FastText', '[BOLD] ELMo', '[BOLD] BERTBASE', '[BOLD] BERTLARGE']
Generally, the biggest improvement by the contextual embeddings over the traditional embeddings is achieved on the Problem type (BERTLARGE: 86.1, GloVe: 77.83). This is reasonable because in clinical notes, diseases and conditions normally appear in certain types of surrounding context with similar grammar structures. Thus, it is necessarily important to take advantage of contextual representations to capture the surrounding context for that particular concept. Interestingly, ELMo outperforms both BERT models for Clinical department and Occurrence.