AdvCodec: Towards A Unified Framework for Adversarial Text Generation
Table 11: Whitebox attack results on BERT-QA in terms of exact match rates and F1 scores by the official evaluation script. The lower EM and F1 scores mean the better attack success rate.
['Model', '[EMPTY]', 'Origin', 'Position Targeted Attack Adv(Sent)', 'Position Targeted Attack Adv(Word)', 'Answer Targeted Attack Adv(Sent)', 'Answer Targeted Attack Adv(Word)', 'Untargeted Attack AddSent', 'Untargeted Attack Adv(scatter)']
We also performed the untargeted scatter attack on QA. We insert 30 random tokens ( but no more than 1/3 the total words of the paragraph) over the paragraph, optimize and find the adversarial tokens that can cause model output the wrong answers in the untargeted manner. We can see the untargeted scatter attack can also achieve a higher untargeted attack success rate than Jia and Liang