Generating Fluent Adversarial Examples for Natural Languages
Table 4: Accuracy results after adversarial training.
['Model', 'Acc (%) Train # = 10K', 'Acc (%) 30K', 'Acc (%) 100K']
To test whether the new models could achieve accuracy gains after adversarial training, experiments are carried out on different sizes of training data, which are subsets of SNLI’s training set. The number of adversarial examples is fixed to 250 during experiment. Adversarial training with w-MHA significantly improves the accuracy on all three settings (with p-values less than 0.02). w-MHA outperforms the genetic baseline with 10K and 30K training data, and gets comparable improvements with 100K training data. Less training data leads to larger accuracy gains, and MHA performs significantly better than the genetic approach on smaller training set.