Rule induction for global explanation of trained models
Table 1: Precision (P), Recall (R), and F-score (F) when explaining neural network predictions using the induced rules with features selected using mutual information (MI) and sensitivity analysis (SA).
['[BOLD] Text class', '[BOLD] MI  [BOLD] P', '[BOLD] MI  [BOLD] R', '[BOLD] MI  [BOLD] F', '[BOLD] SA  [BOLD] P', '[BOLD] SA  [BOLD] R', '[BOLD] SA  [BOLD] F']
We obtain a fidelity score of 0.80 using the proposed technique when the features are pre-selected using the mutual information (MI) score between the transformed inputs and the output labels. Hence, the learned set of if-then-else rules can explain the output of our neural network for 4-class text classification to an F-score of 0.80. The precision of the rules is high, which shows that the rules, if induced, are reliable. The largest F-score of 0.90 is obtained for the class space, and the lowest F-score of 0.75 is obtained for the classes electronics and cryptography. On analyzing the rule-sets obtained for different classes, we see that the complexity of the rules for the space class is lower than that of electronics. For space classification, single terms are often indicative of the correct class. On the other hand, for electronics and cryptography classes, the rules often consist of multiple words, which are jointly used to discriminate between different classes. This suggests that cryptography and electronics are more confusable classes, which is also reflected in their lower F-scores. The observed patterns in the corresponding rule sets are also very similar. Hence, we only present the rules induced using MI feature selection, and those using sensitivity analysis can be found in the Appendix.