Detecting Local Insights from Global Labels: Supervised & Zero-Shot Sequence Labeling via a Convolutional Decomposition
Table 3: Empirical results utilizing exemplar auditing as a decision rule. Token-level positive decisions are only admitted when the test prediction matches that of the exemplar from training. This has the effect of increasing precision across all of the models.
['Model', 'Token-level P', 'Token-level R', 'Token-level  [ITALIC] F0.5']
Across models, this results in an increase in precision. We additionally observed that the average distance to the exemplars was greater for mis-predictions, suggesting that an empirical bound could also be set on allowable distances, which could be particularly useful as another constraint if the evaluation data has the potential to differ significantly from that seen in training. More generally, we found the combination of feature identification from BLADE and feature matching from exemplar auditing to be a useful means of exploring and analyzing the model and data. We provide examples in Appendix B.