TextAT: Adversarial Training with Token-Aware Perturbation for Natural Language Understanding
Table 5: Ablation Studies; GAP is using Global Accumulated Perturbation; DNB is using Discrete Normalization Ball.
['[BOLD] Method  [BOLD] GAP', '[BOLD] Method  [BOLD] DNB', 'RTE', 'MRPC', 'CoLA']
Meanwhile, performances are higher than using regular Frobenius normalization only, which indicates that discrete token normalization method is also working while dealing with in-batch initialized perturbations.