Weight Poisoning Attacks on Pre-trained Models
Table 11: Sentiment Classification Results (SST) for lr=5e-5, batch size=8 (FDK: Full Knowledge, DS: Domain Shift) when pretraining with lr=5e-5
['Setting', 'Method', 'LFR', 'Clean Acc.']
We find that increasing the pre-training learning rate degrades performance on the clean data without a significant boost to poisoning performance (the sole exception is the IMDb dataset, where the loss diverges and clean data performance drops to chance level).