Assigning Medical Codes at the Encounter Levelby Paying Attention to Documents
Table 1: Document-level F1-score calculated by comparing document attention from ELDAN and human coders on 20 CPT codes. #enc is the number of encounters that contains the code. #doc is the number of documents within those encounters. #source is the number of documents being labeled by human coders as the source documents for the code. Attention (from ELDAN) and Chance both report document-level F1-score, and Diff is the difference between them.
['CPT Codes', '#enc', '#doc', '#source', 'Attention', 'Chance', 'Diff']
for the most frequent 20 encounter-level codes, with surprisingly strong results: 100% F1-score on 7 out of 19 available codes. However, even chance performance could be good if the number of possible documents to assign credit to is very small. Therefore we compare to the chance baseline. ELDAN is consistently better, usually by a large margin. These results support the conclusion that ELDAN’s document attention is effective in identifying signal from “source” documents for the targeted code — crucially, without training on document-level annotations.