SnapToGrid: From Statistical to Interpretable Models for Biomedical Information Extraction
Table 2: Performance of the statistical model with L2-regularized LR, using only features that can be converted to rules.
['Event Class', 'Recall', 'Precision', 'F1']
This configuration generated 1,190,029 features with non-zero weights. The table shows that this model achieved an overall F1 score of over 40 points, which likely puts it in the top 5 or 6 (out of 24) systems that participated in the actual challenge. For a fair comparison, we next trained the same model but using only features that can be converted to rules. As discussed, the features that were removed include bag-of-word features and features that count occurrences of tokens or entities in text. This suggests that rule languages need to be extended if they are to have the same representational power as feature-based models. Given that the focus of this work is not on the design of rule-based languages for IE, we will use this latter model as the starting point of our approach, ignoring (for now) the performance penalty observed above.