Temporal Self-Attention Network for Medical Concept Embedding
TABLE VII: Mortality prediction performance on MIMIC III dataset.
['[BOLD] Model', '[BOLD] test loss', '[BOLD] test PR-AUC', '[BOLD] test ROC-AUC']
We find that TeSAN again consistently outperforms all baseline models. Achieving high specificity in mortality prediction is relatively easy as there are many more negative samples than positive ones. However, correctly identifying positive cases while ignoring negative ones requires a model differentiate between positive cases. This means attending to the details of patient records, such as the relationship between the diagnosis codes and temporal intervals. This is why TeSAN demonstrates a significant improvement in PR-AUC and ROC-AUC. Also, we note that GloVe shows very poor PR-AUC and ROC-AUC, and we observe that medical concepts with low frequencies are assigned near-zero vectors in this model, which might explain its poor performance.