Learning to Discriminate Perturbations forBlocking Adversarial Attacks in Text Classification
Table 7: The performance of DISP using ground-truth and recovered tokens over different types of attacks in SST-2. Result are in accuracy. Note that DISPG denotes DISP using ground-truth tokens.
['Method', 'Insertion', 'Delete', 'Swap']
Embedding Estimator. Although DISP is not required to recover the ground-truth perturbed tokens, the embedding estimator plays an important role to derive appropriate embedding vectors that obtain the original semantics. We first evaluate the performance of embedding estimator as a regression task. The RMSE scores of estimated embeddings are 0.0442 and 0.1030 in SST-2 and IMDb datasets, which are small enough to derive satisfactory tokens. To further demonstrate the robustness of the embedding estimator and estimated embeddings, we identify the perturbations with our discriminator and replace them with the ground-truth tokens. DISP and DISPG denotes the recovery performance with our estimator and gound-truth tokens, respectively. More specifically, the accuracy of DISPG presents the upperbound performance gained by the embedding estimator. The experimental results demonstrate the robustness of the embedding estimator while the estimated embeddings only slightly lower the accuracy of DISP.