Pathologies of Neural Models Make Interpretations Difficult
Table 3: Human Accuracy increases after fine-tuning the models. Humans also prefer gradient-based reduced examples over randomly reduced ones, indicating that the reduced examples are more meaningful to humans after regularization.
['[EMPTY]', 'Accuracy Before', 'Accuracy After', 'vs.\xa0Random Before', 'vs.\xa0Random After']
We also repeat the vs. Random experiment : we re-generate the random examples to match the lengths of the new reduced examples from input reduction, and find humans now prefer the reduced examples to random ones. The increase in both human performance and preference suggests that the reduced examples are more reasonable; model pathologies have been mitigated.