NRC-Canada at SMM4H Shared Task: Classifying Tweets Mentioning Adverse Drug Reactions and Medication Intake
Table 7: Task 2: Results of our best system (submission 1) on the test set when one of the feature groups is removed.
['[BOLD] Submission', '[ITALIC] Pclass1+ [ITALIC] class2', '[ITALIC] Rclass1+ [ITALIC] class2', '[ITALIC] Fclass1+ [ITALIC] class2']
In this task, the general textual features (row b) played a bigger role in the overall performance than the domain-specific (row c) or sentiment lexicon (row d) features. Removing this group of features results in more than 2.5 percentage points drop in the F-measure affecting both precision and recall (row b). However, removing any one feature subgroup in this group (e.g., general n-grams, general clusters, general embeddings, etc.) results only in slight drop or even increase in the performance (rows b.1–b.4). This indicates that the features in this group capture similar information. Among the domain-specific features, the n-grams generalized over domain terms are the most useful. The model trained without these n-grams features performs almost one percentage point worse than the model that uses all the features (row c.1). The sentiment lexicon features were not helpful (row d).