Frequency-Guided Word Substitutions for Detecting TextualAdversarial Examples
Table 6: Classification accuracies before and after applying FGWS and DISP to the clean test sets.
['[BOLD] Dataset', '[BOLD] Classifier', '[BOLD] Clean', '\\textsc  [ITALIC] FGWStest', '\\textsc  [ITALIC] DISPtest']
While FGWS shows to aid in detecting adversarial sequences, such transformations might still result in semantic shifts of the manipulated sequences and hence a decrease in model classification performance on unperturbed sequences after transformation. We explicitly investigate this by transforming the sequences in the test sets using FGWS (with δ set to the same values as for the detection task), and evaluate the models’ classification accuracies on the test sets after transformation. For comparison, we also report the accuracies after transforming the test sets using DISP (denoted \textscDISPtest). Here, one can observe that FGWS has only little influence on classification performance on the IMDb dataset and even leads to slight improvements, whereas slight decreases are observed on \textscDISPtest. On SST-2, notable performance decreases can be observed with respect to both DISP and FGWS, although the decreases are more dominant on the data transformed using FGWS.