Hybrid Text Feature Modeling for Disease Group Prediction using Unstructured Physician NotesSubmitted to ICCS 2020.
Table 3: Experimental Results – Baseline Comparison
['[BOLD] Parameter', '[BOLD] Proposed Approach', '[BOLD] Bag-of-words (TF-IDF)', '[BOLD] Only Word2Vec', '[BOLD] Only Pre-trained']
For evaluating the proposed hybrid text feature modeling approach, we designed experiments to compare its performance against that of baseline feature modeling approaches – TF-IDF based bag-of-words approach, trained word embedding approach (only Word2Vec model) and a pre-trained word embedding approach (using word embedding model trained on PubMed, PMC and Wikipedia English articles). The bag-of-words approach is based on calculated term Frequency and inverse document frequency ( tfXidf) scores determined from the physician notes corpus. The Sklearn English stopword list was used to filter the stopwords and n-gram (n=1,2,3) features were considered. Finally, the top 1000 features were extracted from the corpus and then fed into the neural network model for training. The other two baselines were kept the same as explained in Section 2. It is to be noted that in the neural network configuration, only one input is present, i.e., there is no hybrid weighted addition layer. It can be observed that the proposed approach that involves a hybrid weighted combination of pre-trained and trained word embeddings is able to perform comparatively better in terms of all metrics. Dicussion: From our experiments, we observed a significant potential in developing prediction based CDSS using unstructured text reports directly, eliminating the dependency on the availability of structured patient data and EHRs. The proposed approach that involves a textual feature modeling and a neural network based prediction model was successful in capturing the rich and latent clinical information available in unstructured physician notes, and using it to effectively learn disease group characteristics for prediction. The Word2Vec model was trained on the physician notes corpus with optimized parameter configuration generates effective word embedding features to be fed into the neural network model. The hybrid combination of these with the embedding features of the corpus generated by the pre-trained model trained on PubMed and PMC articles further enhanced and enriched the semantics of the textual features with biomedical domain knowledge.