Assigning Medical Codes at the Encounter Levelby Paying Attention to Documents
Figure 1: Left: Encounter-level F1-scores of the 20 most frequent CPT codes. #Docs is the average number of documents found in the encounters that contain the code; prevalence is the percentage of all encounters that contain that code. Right: Macro average of encounter-level F1 scores for every 10 codes (from most to least frequent). ΔELDAN =\textscELDAN+transfer−\textscELDAN.
['Average', 'Prevalence', 'ELDN', 'ELDAN', 'ELDAN +transfer', 'ΔELDAN']
To show the trend across the full range of codes we macro-average every 10 codes from most frequent to least frequent ELDAN with or without naïve transfer learning consistently outperforms ELDN, even for extremely rare codes (<0.1%). As codes become rarer, ELDAN+transfer tends toward outperforming ELDAN more substantially; see increasing trend for ΔELDAN. This improvement can be explained by viewing the embedding layer as a vector space model that maps sparse features that are extracted from the document (such as medical concepts, UMLS CUIs) to a dense representation, which can be effective for bootstrapping the training of rare codes. Our 80-10-10 dataset split results in 371,092 encounters for training, 46,387 encounters for development/tuning, and 46,387 encounters for testing. Note that no document-level annotations are available. Since we are in an imbalanced setting (some medical codes can be extremely rare, see Fig. No resampling is done for the development set and test set. These hyperparameters were selected based on our results on the development set.