Large-Scale Adversarial Training for Vision-and-Language Representation Learning
Table 7: Ablation study on Villa-pre (pre-training) and Villa-fine (finetuning) with large model size.
['Method', 'VQA test-dev', 'VCR (val) Q→A', 'VCR (val) QA→R', 'VCR (val) Q→AR', 'Ave.']
On average, adversarial pre-training and finetuning bring +1.48 and +2.21 performance gain, respectively. Combining the two AT stages provides further improvement.