BAE: BERT-based Adversarial Examples for Text Classification
Table 5: A denotes the % of test instances which are successfully attacked by BAE-R/I, but not BAE-R , i.e, ((R/I)∩¯¯¯R) ; B : (R/I)∩¯¯¯I ; C : (R/I)∩¯¯¯R∩¯¯¯I
['[BOLD] Dataset', '[BOLD] Word-LSTM A', '[BOLD] Word-LSTM B', '[BOLD] Word-LSTM C', '[BOLD] Word-CNN A', '[BOLD] Word-CNN B', '[BOLD] Word-CNN C', '[BOLD] BERT A', '[BOLD] BERT B', '[BOLD] BERT C']
Ablation Study We analyze the benefits of R/ From the table, the splits A and B are the % of test points which compulsorily need I and R operations respectively for a successful attack. We can observe that the split A is larger than B thereby indicating the importance of the I operation over R. Test points in split C require both R and I operations for a successful attack. I operations. Thus, this study gives positive insights towards the importance of having the flexibility to both replace and insert words.