Convolutional Neural Networks for Medical Diagnosis from Admission Notes
Table 3: Evaluation metrics on individual disease classification. The ten class labels correspond to coronary artery disease, hemorrhage, pneumonia, myocardial infarction, gastrointestinal bleeding, fracture, aortic stenos, cardiac failure, prematurity and stroke. The evaluation metrics ACC, TNR, FPR, FNR refer to accuracy, true positive rate, false positive rate, false negative rate. The mapping between label indices and diseases are: 0-coronary artery disease, 1-hemorrhage, 2-pneumonia, 3-myocardial infarction, 4-gastrointestinal bleeding, 5-fracture, 6-aortic stenosis, 7-cardiac failure, 8-prematurity, 9-stroke. The standard errors computed by 5 set of results of each model using different random splitting are appended after the metric values with a ± sign.
['Model', 'Label', 'ACC', 'TNR', 'FPR', 'FNR', 'Precision', 'Recall', 'F1']
Furthermore, although some models have relatively high F1 score in categories with large sample size (e.g., coronary artery disease, hemorrhage, pneumonia), they performed badly on categories with less samples (e.g., myocardial infarction, aortic stenosis, prematurity). For example, the SVM achieved an F1 score value of 80.09% for the hemorrhage class, only around 4% lower than the F1 score value of the CNN model, it was only able to achieve F1 score values of 15.02% on aortic stenosis and 19.85% on prematurity. On the contrary, the CNN model performed equally well on all disease classes independently of frequently they appear and are diagnosed in the MIMIC corpus. The highest F1 score values achieved from any of the baseline models for aortic stenosis and prematurity are 42.80% and 42.85% respectively. Among all the baseline models, the logistic regression and the MLP model achieved less than 5% F1 score for the prematurity class. On the other side, the CNN model still achieved F1 score values of 61.29% and 65.04% on the same diseases. This fact further confirms that the proposed CNN model works remarkably well at classifying rare (not appearing frequently in the corpus) diseases.