Defense against Adversarial Attacks in NLP via Dirichlet Neighborhood Ensemble
Table 3: Natural language inference on SNLI dataset.
['[BOLD] SNLI', '[BOLD] BOW  [BOLD] CLN', '[BOLD] BOW  [BOLD] PWWS', '[BOLD] BOW  [BOLD] GA-LM', '[BOLD] BOW  [BOLD] GA', '[BOLD] DecomAtt  [BOLD] CLN', '[BOLD] DecomAtt  [BOLD] PWWS', '[BOLD] DecomAtt  [BOLD] GA-LM', '[BOLD] DecomAtt  [BOLD] GA', '[BOLD] BERT  [BOLD] CLN', '[BOLD] BERT  [BOLD] PWWS', '[BOLD] BERT  [BOLD] GA-LM', '[BOLD] BERT  [BOLD] GA']
Although our proposed baseline RAN (k=16) achieves a slightly higher accuracy (just 1\% difference) with BERT under PWWS attack, it’s accuracy rapidly drops to 27\% under the more sophisticated attack GA, where DNE still yields 61.6\% in accuracy. The results on SNLI show that DNE can be applied to attention-based models like DecomAtt and scale well to large architectures such as BERT. We leave the results of IPB with BERT as unknown because there is still a question whether IBP-based method can be applied to BERT.