Large-Scale Adversarial Training for Vision-and-Language Representation Learning
Table 5: Results on LXMERT with Villa-fine (finetuning).
['Method', 'VQA test-dev', 'VQA test-std', 'GQA test-dev', 'GQA test-std', 'Meta-Ave.']
Results on LXMERT Villa is a generic framework that can be readily applied to any V+L models. To demonstrate its generalization ability, we conduct additional experiments using LXMERT as the backbone. Since adversarial pre-training is highly time-consuming, we only focus on adversarial finetuning for LXMERT. We use VQA and GQA as the evaluation tasks, the same as LXMERT. Compared to LXMERT, Villa-fine achieves higher accuracy on validation set and lower accuracy on training set for both VQA and GQA, clearly demonstrating its regularization effect in preventing overfitting of large-scale models. Results of LXMERT with Villa-fine on NLVR2 In the LXMERT paper, LXMERT has been applied to VQA, GQA, and NLVR2 tasks. Villa-fine instantly provides +0.98 performance boost without changing any model architectures and hyper-parameter settings used in LXMERT. Results on GQA