SnapToGrid: From Statistical to Interpretable Models for Biomedical Information Extraction
Table 6: Performance of the rule-based model, after expert intervention.
['Event Class', 'Recall', 'Precision', 'F1']
The table shows that most of the F1 loss has been recovered: the overall F1 score for this system approaches 35 F1 points, and is less than 1 F1 point behind the L1-regularized LR statistical model. In addition of reducing the number of rules in the model, the experts’ recommendations increased recall by over 4%, which is more than what was lost during the conversion to rules. However, the precision of this configuration decreased by 11%, which we blame on the experts’ limited familiarity with the BioNLP task, and the strict settings of the experiment (no access to data, limited time). However, all in all, this experiment demonstrates that the rule-based model produced by the proposed approach is interpretable: the experts understood the model, and were able to improve it, both with respect to its generalization power and its readability.