Clinical Concept Extraction for Document-Level Coding
Table 2: Test set results using the augmentation methods.
['[BOLD] Model', '[BOLD] AUC Macro', '[BOLD] AUC Micro', '[BOLD] AP Macro', '[BOLD] AP Micro', '[BOLD] F1 Macro', '[BOLD] F1 Micro', '[BOLD] R@k 8', '[BOLD] R@k 15', '[BOLD] P@k 8', '[BOLD] P@k 15']
Overall, the concept-augmented models are indistinguishable from the baseline, and there is no significant difference between annotation type or recombination method, although the linear combination method with ICD9 annotations is the best performing and rivals the baseline. We separate and analyze the two distinct components of cTAKES’ annotation ability for further analysis: 1) how well cTAKES recognizes the location of concepts in the text (NER), and 2) how accurately cTAKES maps the recognized positions to the correct clinical concepts (ontology mapping). Annotation sparsity (NER) We investigate these hypotheses here. We evaluate performance of ablations relative to the augmentation model and baseline to determine whether each component individually adds value. The ablations are: