Aligning Faithful Interpretations with their Social Attribution
Table 4: The percentage increase in error of selector-predictor highlight methods compared to an equivalent architecture model which was trained to classify complete text. E.g., 10% means that the error is x1.1, or 110%, that of the full-text equivalent of the same architecture. We prioritize the numbers reported in previous work whenever possible, and preferably the original papers of each methodology (italics means our results). Architectures are not necessarily consistent across different cells in the table, and thus they do not imply performance superiority of absolute metrics with the “best” architectures. The highlight lengths chosen for each experiment were chosen with precedence whenever possible, and otherwise chosen as 20% following jain2020 precedence.
['[EMPTY]', 'SST-2', 'SST-3', 'SST-5', 'LGD', 'AG News', 'IMDB', 'Ev. Inf.', 'MultiRC', 'Movies', 'Beer']
In many cases, this decrease is severe. Although this phenomenon is treated as a reasonable and matter-of-fact necessity by literature on highlight interpretations and rationales, the intuition behind it is not trivial to us.