Deep Neural Models for Medical Concept Normalization in User-Generated Texts
Table 2: The performance of the proposed models and the state-of-the-art methods in terms of accuracy.
['[BOLD] Method', '[BOLD] CADEC Random', '[BOLD] CADEC Custom', '[BOLD] PsyTAR Random', '[BOLD] PsyTAR Custom', '[BOLD] SMM4H Official']
We evaluate our models based on classification accuracy, averaged across randomly divided five folds of the CADEC and PsyTAR corpora. For SMM4H 2017 data, we adopted the official training and test sets Sarker et al. Therefore, we set up a specific train/test split procedure for 5-fold cross-validation on the CADEC and PsyTAR corpora: we removed duplicates of mentions and grouped medical records we are working with into sets related to specific medical codes. Then, each set has been split independently into k folds, and all folds have been merged into the final k folds named Custom KFolds. Custom folds of CADEC are adopted from our previous work Tutubalina et al. We have also implemented a simple baseline approach that uses exact lexical matching with lowercased annotations from the training set.