Feature-Augmented Neural Networks for Patient Note De-identification
Table 3: Binary token-based results (%) . The reported results are optimized by recall, and averaged over 5 runs. The symbol 1 indicates that the PHI type is not required by HIPAA. The PHI type “location” designates any location that is not a street name, zip code, state or country. P stands for precision, R for recall, and F1 for F1-score.
['[EMPTY]', 'No feature P', 'No feature R', 'No feature F1', 'EHR features P', 'EHR features R', 'EHR features F1', 'All features P', 'All features R', 'All features F1', 'Support']
The system using only the EHR features yields the highest recall for 6 out of 12 PHI types. Most importantly, the recall for patient and doctor names are higher when using features than when using no feature: this is expected as the patient name of the note and the doctor names are used as features. In fact, the two remaining false negatives for patient names are annotation errors. For example, in the sentence “The patient responded well to Natrecor in the past, but the improvement disappeared soon”, the drug name Natrecor was incorrectly marked as a patient name by the human annotator.