A Multi-Task Learning Framework for Extracting Drugs and Their Interactions from Drug Labels
Table 3: Preliminary results based on 11-fold cross validation over Training-22 with two held-out drug labels per fold. When NLM-180 is incorporated, the training data used for each fold consists of 20 non-held out drug labels from Training-22 and all 180 drug labels from NLM-180.
['[BOLD] Model / Data', 'Entity (Primary)  [BOLD] P (%)', 'Entity (Primary)  [BOLD] R (%)', 'Entity (Primary)  [BOLD] F (%)', 'Relation (Primary)  [BOLD] P (%)', 'Relation (Primary)  [BOLD] R (%)', 'Relation (Primary)  [BOLD] F (%)']
Evaluations were produced as as result of 11-fold cross-validation over Training-22 with two drug labels per fold. Instead of macro-averaging over folds, and thereby weighting each fold equally, we evaluate on the union of all 11 test-fold predictions. Lowered recall is mostly due to simplifying assumptions; e.g., we only consider non-overlapping mentions. For coordinated disjoint cases such as “X and Y inducers”, we only considered “Y inducers” in our simplifying assumption. Imperfect precision is due to discrepancies between the tokenization scheme used by our method and that used to produce gold annotations; this leads to the occasional mismatch in entity offsets during evaluation. Adding character-CNN based word representations improved performance marginally, more so for NER than relation extraction We also implemented several tweaks to the pre-processing and post-processing aspects of the model based on preliminary error analysis including (1) using drug class mentions (e.g., “diuretics”) as proxies if the drug label is not mentioned directly; (2) removing modifiers such as moderate, strong, and potent so that output conforms to official annotation guidelines; and (3) purging predicted mentions with only stopwords or generic terms such as “drugs” or “agents.”