Masking as an Efficient Alternative to Finetuningfor Pretrained Language Models
(a) Masking
['TrainEval', 'SST2', 'SEM']
We use the two sentiment classification datasets in our task pool: SST2 and SEM. We simply evaluate the model masked or finetuned on SST2 against the dev set of SEM and vice versa. For example, in (a), cell value -13.4 means that the SST2-masked BERT performs 13.4% worse than the majority baseline