Language Models Are An Effective Representation Learning Technique For Electronic Health Record Data
Table 4: AUROC of prediction models with different language modeling objectives
['Outcome Name', 'CLMBR Language Model', 'DoctorAI Language Model', 'Difference']
In order to measure the benefits of CLMBR’s more complicated language model we implemented the DoctorAI language model as described in the methods and used it to construct representations. We then trained clinical prediction models using representations from the two different language models. We observed a consistent improvement in performance across all outcomes with the complicated language modeling objective, with a larger improvement for two of the five outcomes (ICU transfer, abnormal HbA1c).