Obtaining Faithful Interpretations from Compositional Neural Networks
Table 2: Faithfulness and performance scores for various NMNs on DROP. ∗lower is better. †min-max is average faithfulness of find-min-num and find-max-num; find-arg of find-num and find-date.
['[BOLD] Model', '[BOLD] Performance (F1 Score)', '[BOLD] Overall Faithful. (cross-entropy∗ ↓)', 'Module-wise Faithfulness∗ (↓) find', 'Module-wise Faithfulness∗ (↓) filter', 'Module-wise Faithfulness∗ (↓) relocate', 'Module-wise Faithfulness∗ (↓) min-max†', 'Module-wise Faithfulness∗ (↓) find-arg†']
Similar to Visual-NMN, this shows that supervising intermediate modules in a program leads to better faithfulness.