Clinical Concept Extraction for Document-Level Coding
Table 9: The effect of tuning λ on dev set performance on the ICD9 coding task, for the pre-convolution model with a linear auxiliary layer and ICD9 annotations. We select λ=1 for reporting test results; there isn’t a clear value which produces strictly better performance.
['[ITALIC] λ', '[BOLD] AUC Macro', '[BOLD] AUC Micro', '[BOLD] AP Macro', '[BOLD] AP Micro', '[BOLD] F1 Macro', '[BOLD] F1 Micro', '[BOLD] R@k 8', '[BOLD] R@k 15', '[BOLD] P@k 8', '[BOLD] P@k 15', '[BOLD] Auxiliary Tagging Accuracy After one epoch', '[BOLD] Auxiliary Tagging Accuracy After last epoch']
Notably, for even very small values of λ, we achieve very high validation accuracy on the auxiliary task. This performance does not change with larger weightings, indicating that the auxiliary task may not be difficult enough to result in effective knowledge transfer.