Closing the Gap: Joint De-Identification and Concept Extraction in the Clinical Domain
Table 5: Pipeline analysis results on Spanish concept extraction. “non-anon.” indicates the original text without anonymization; “anon-gold” and “anon-predicted” refer to texts with replacements for gold/predicted de-identification labels, respectively.
['[EMPTY]', 'train on', 'test on', 'dev', 'test']
Finally, we analyze the impact of de-identification on CE. Thus, we can also investigate the difference between gold and predicted de-identification labels. The CE model benefits from being trained and evaluated on anonymized data (lines 4-6). However, it hurts to train on non-anonymized data and evaluate on predicted de-identification labels (line 1 vs. 2) and vice versa (line 1 vs. 3). This supports our motivation that it is necessary to investigate anonymization and downstream applications together. The difference of training on gold vs. predicted de-identification labels (lines 4-6) is only marginal, suggesting that state-of-the-art de-identification systems are good enough to be used in such settings.