Elephant in the Room: An Evaluation Framework for Assessing Adversarial Examples in NLP
Table 1: Results based on automatic metrics. Top half (A) presents 3 different target classifiers evaluated on one dataset (yelp50); bottom half (B) tests 3 datasets using one classifier (BiLSTM+A). For ACPT, less negative values are better. Boldface indicates optimal performance for an attacking performance and target classifier. Missing numbers (dashed lines) indicate the method is unable to produce the desired accuracy, e.g. HotFlip with only 1 word flip produces 81.5% accuracy (T1) when attacking CNN on yelp50, and so T0 accuracy is unachievable.
['(A) Models: acc', '(A) Models: acc', 'Dataset: yelp50 BiLSTM+A: 96.8', 'Dataset: yelp50 BiLSTM+A: 96.8', 'Dataset: yelp50 BiLSTM+A: 96.8', 'Dataset: yelp50 BiLSTM+A: 96.8', 'Dataset: yelp50 CNN: 94.3', 'Dataset: yelp50 CNN: 94.3', 'Dataset: yelp50 CNN: 94.3', 'Dataset: yelp50 CNN: 94.3', 'Dataset: yelp50 BiLSTM: 96.6', 'Dataset: yelp50 BiLSTM: 96.6', 'Dataset: yelp50 BiLSTM: 96.6', 'Dataset: yelp50 BiLSTM: 96.6']
To check how well these adversarial examples generalise to fooling other classifiers, also known as transferability, we feed the adversarial examples from the 3 best methods, i.e. TYC, HotFlip and TextFooler, to a pre-trained BERT trained for sentiment classification and measure its accuracy Interestingly, we find that TextFooler, the best performing method, produces the least effective adversarial examples for BERT, indicating poor transferability, while examples generated from TYC perform the best in fooling BERT. Manually inspecting the examples generated by TextFooler, we notice it tries to replace as few words as possible to move an examples just across the decision boundary of the target classifier (indicated by very close scores between different labels). This change appears be very targeted for a specific classifier, and as such is unlikely to fool other classifiers.