Oscar: Object-Semantics Aligned Pre-training for Vision-Language Tasks
Table 2: Overall results on six tasks. Δ indicates the improvement over SoTA. SoTA with subscript S, B, L indicates performance achieved by small models, VLP of similar size to BERT base and large model, respectively. Most results are from [5], except that image captioning results are from[11, 47], NoCaps results are from [1], VQA results are from [39].
['Task', 'Image Retrieval R@1', 'Image Retrieval R@5', 'Image Retrieval R@10', 'Text Retrieval R@1', 'Text Retrieval R@5', 'Text Retrieval R@10', 'Image Captioning B@4', 'Image Captioning M', 'Image Captioning C', 'Image Captioning S', 'NoCaps C', 'NoCaps S', 'VQA test-std', 'NLVR2 test-P']
For all the tables in this paper, Blue indicates the best result for a task, and gray background indicates results produced by Oscar. As shown in the table, our base model outperforms previous large models on most tasks, often by a significantly large margin. It demonstrates that the proposed Oscar is highly parameter-efficient, partially because the use of object tags as anchor points significantly eases the learning of semantic alignments between images and texts. Note that Oscar is pre-trained on 6.5 million pairs, which is less than 9.6 million pairs used for UNITER pre-training and 9.18 million pairs for LXMERT.