Frequency-Guided Word Substitutions for Detecting TextualAdversarial Examples
Table 3: Attack success rates (%) of the low-frequency attack.
['[BOLD] Dataset', '[BOLD] Classifier', '\\textsc  [ITALIC] Frequencyr', '\\textsc  [ITALIC] Frequencyp']
We adhere to previous experiments by allowing 20% of word changes made by the attack. We observe that while \textscFrequencyr exhibits poor attack performances, \textscFrequencyp achieves to misclassify the majority of sequences for the IMDb dataset as well as an increased amount of sequences on SST-2. These findings indicate that while all three models seem to be robust against random low-frequency substitutions, adding a simple selection heuristic for more impactful word replacements yields strong attack performance increases. This clearly shows that, although the word frequency differences exist across a variety of attacks, relying on this heuristic alone does not suffice to confidently lead the investigated classification models into making false predictions.