Weight Poisoning Attacks on Pre-trained Models
Table 5: Hyperparameter Change Effects (SST-2, full knowledge).
['Hyperparameter change', 'LFR', 'Clean Acc.']
We examine the effect of changing various hyperparameters on the SST-2 dataset during fine-tuning for RIPPLES. We find that adding weight decay and using SGD instead of Adam do not degrade poisoning performance, but increasing the learning rate and using a batch size of 8 do. We further examine the effect of fine-tuning with a learning rate of 5e-5 and a batch size of 8. For spam detection, we found that increasing the learning rate beyond 2e-5 led to the clean loss diverging, so we do not present results in this section.