Gradient-Based Adversarial Training on Transformer Networks for Detecting Check-Worthy Factual Claims
Table 2. Perturbation Combinations Study Results Averaged Across Stratified 4-Fold Cross Validation
['(l)2-7  [BOLD] ID', '[BOLD] P  [BOLD] NCS', '[BOLD] P  [BOLD] CFS', '[BOLD] R  [BOLD] NCS', '[BOLD] R  [BOLD] CFS', '[BOLD] F1  [BOLD] NCS', '[BOLD] F1  [BOLD] CFS']
From the results we conclude that setting 6 produces the best models for our task. Particularly, this setting produces the best recall for the CFS class, which is arguably the more important class. The sacrifice in recall, with respect to the NCS class, compared to other settings is only ≈0.02 at most. While setting 4 achieves the best performance in precision with respect to the CFS class, the drop in recall for the CFS class makes it not viable. Thus, from here on, any results dealing with adversarial training will employ setting 6 and perturb only the tok embedding layer. cs_train_steps: number of epochs to run cs_lr: learning rate during optimiation cs_perturb_norm_length: norm length of adversarial perturbation cs_kp_cls : keep probability of dropout in fully connected layer cs_lambda: adversarial loss coefficient (eq. cs_combine_reg_adv_loss: