Elephant in the Room: An Evaluation Framework for Assessing Adversarial Examples in NLP
Table 2: Computational time (seconds per example) of TYC, HotFlip and TextFooler when attacking different datasets.
['Methods', 'rt', 'yelp50', 'yelp200']
As an additional insight, we also evaluated the computational time of different attacking methods. We calculate the the generation time of the three best performing methods TYC, HotFlip and TextFooler when they attack the test dataset of rt, yelp50, and yelp200 at T2 attacking threshold. We found that HotFlip achieves comparable performance with TextFooler but only consumes 1/3 of its computational time. TYC is even more time-consuming than TextFooler. Also, the speed of HotFlip seems not affected by the increase of the sentence lengths, but TextFooler and TYC take much longer time to attack longer input examples.