Defense of Word-level Adversarial Attacks via Random Substitution Encoding
Table 2: The evaluation results under different settings.
['Dataset', 'Attack Model', 'LSTM (%) NT', 'LSTM (%) AT', 'LSTM (%) RSE', 'Bi-LSTM (%) NT', 'Bi-LSTM (%) AT', 'Bi-LSTM (%) RSE', 'Word-CNN (%) NT', 'Word-CNN (%) AT', 'Word-CNN (%) RSE']
For each base model with each dataset, we highlight the highest classification accuracy for different defense frameworks in bold to indicate the best defense performance. But under other attack models, our RSE framework can get the best accuracy. For each column in each box, target models have the lowest accuracy under PWWS attack, which demonstrates PWWS is the most effective attack model. The accuracy of NT and AT drop significantly under PWWS attack. But RSE has the best defense performance since the accuracy loss is very small compared with ‘No attack’. Under different settings (various datasets, attack models and base models), our RSE framework has a better performance with few accuracy decrease. This demonstrates the generalization of RSE framework to strengthen a robust deep neural network against synonym adversarial attacks.