Feature-Augmented Neural Networks for Patient Note De-identification
Table 2: Binary HIPAA token-based results (%) for the ANN model, averaged over 5 runs. The metric refers to the detection of PHI tokens versus non-PHI tokens, amongst PHI types that are defined by HIPAA. “No feature” is the model utilizing only character and word embeddings, without any feature. “EHR features” uses only 4 features derived from EHR database: patient first name, patient last name, doctor first name, and doctor last name. “All features” makes use of all features, including the EHR features as well as other engineered features listed in Table 1. “Optimized by F1-score” and “optimized by recall” means that the epochs for which the results are reported are optimized based on the highest F1-score or the highest recall on the validation set, respectively.
['[EMPTY]', 'Binary HIPAA (optimized by F1-score) Precision', 'Binary HIPAA (optimized by F1-score) Recall', 'Binary HIPAA (optimized by F1-score) F1-score', 'Binary HIPAA (optimized by recall) Precision', 'Binary HIPAA (optimized by recall) Recall', 'Binary HIPAA (optimized by recall) F1-score']
The epochs for which the results are reported are optimized based on either the highest F1-score or the highest recall on the validation set. As expected, choosing the epoch based on the recall improves the recall on the test set, while lowering the precision. Overall, adding features consistently improves the results.