Towards Hierarchical Importance Attribution: Explaining Compositional Semantics for Neural Sequence Models
Table 1: Correlation between word & phrase importance attribution and linear model coefficients & SST-2 human annotations, achieved by baselines and our explanation algorithms.
['[BOLD] Dataset Model', '[BOLD] SST-2 BERT', '[BOLD] SST-2 BERT', '[BOLD] SST-2 LSTM', '[BOLD] SST-2 LSTM', '[BOLD] Yelp Polarity BERT', '[BOLD] Yelp Polarity LSTM', '[BOLD] TACRED BERT', '[BOLD] TACRED LSTM']
Generally, explanation algorithms that follow our formulations achieve highest word ρ and phrase ρ for all the datasets and models. SOC and SCD perform robustly on the deep Transformer model, achieving higher word ρ and phrase ρ than input occlusion and contextual decomposition algorithms by a large margin. We see the simple Direct Feed method provide promising results on shallow LSTM networks, but fail in deeper Transformer models. The statistic based approximation of the context independent importance, which do not employ a trained sampler, yields competitive words ρ but low phrase ρ. Our analysis show that it is common that a long phrase does not exist in previously seen examples. In this case, phrase ρ achieved by statistic based approximation is pushed towards that of the input occlusion algorithm. For computing context independent importance of a phrase, an intuitive and simple alternative approach, which is nevertheless neglected in prior literature, is to only feed the input to the model and treat the prediction score as the explanation.