Drug-Drug Interaction Extraction from Biomedical Text Using Long Short Term Memory Network
Table 4: Performance improvement after filtering negative instance from the dataset. △ indicates percentage of relative improvement in F1 score.
['[BOLD] Models', '[BOLD] Before  [BOLD] Precision', '[BOLD] Before  [BOLD] Recall', '[BOLD] Before  [BOLD] F1 Score', '[BOLD] After  [BOLD] Precision', '[BOLD] After  [BOLD] Recall', '[BOLD] After  [BOLD] F1 Score', '△']
Following our filtering rules, a large number of negative instances along with few positive instances are removed from the dataset. However none of the positive instances from test set are removed. There was a significant change in imbalance ratios of interaction classes due to the filtering step. For example, imbalance ratio changed from 1:32.6 to 1:19.1 for the interaction class Advice and from 1:146 to 1:116.8 for the interaction class Int. All models when using filtered dataset gave improved performance in terms of F1 score. All three LSTM models obtained more than 3% of relative improvement with the use of filtered dataset. Performance improvement can be attributed to two factors: cleaner data and reduction of imbalance. Both these factors are result of the data filtering step.