BERT-XML: Large Scale Automated ICD Coding Using BERT Pretraining
Table 3: Model Performance - Low Frequency ICDs
['[EMPTY]', '[BOLD] Macro AUC']
In addition, while the BERT based models do better than standard models on average, we see very pronounced gains in lower frequency ICDs. Note that the best non-BERT model does worse on this set compare to its performance on all ICDs, while the best BERT model performs better on average on the lower frequency ones. This further illustrates the value of the unsupervised pretraining and provides good motivation to expand our analysis to even less frequent ICD codes in future work.