Build it Break it Fix it for Dialogue Safety:Robustness from Adversarial Human Attack
Table 6: Test performance of best standard models trained on standard task rounds (models Si for each round i) and best adversarial models trained on adversarial task rounds (models Ai). All models are evaluated using offensive-class F1 on each round of both the standard task and adversarial task. A0 is the baseline model trained on the existing Wiki Toxic Comments (WTC) dataset. Adversarial models prove to be more robust than standard ones against attack (Adversarial Task 1-3), while still performing reasonably on Standard and WTC tasks.
['Task Type', 'Task Round', 'WTC Baseline  [ITALIC] A0', '[ITALIC] Standard models  [ITALIC] S1', '[ITALIC] Standard models  [ITALIC] S2', '[ITALIC] Standard models  [ITALIC] S3', '[ITALIC] Adversarial models  [ITALIC] A1', '[ITALIC] Adversarial models  [ITALIC] A2', '[ITALIC] Adversarial models  [ITALIC] A3']
5.2.2 Fix it Phase The adversarially trained models Ai prove to be more robust to adversarial attack: on each round of adversarial testing they outperform standard models Si.