UNITER: UNiversal Image-TExt Representation Learning
Table 3: Evaluation on pre-training tasks and datasets using VQA, Image-Text Retrieval on Flickr30K, NLVR2, and RefCOCO+ as benchmarks. All results are obtained from UNITER-base. Averages of R@1, R@5 and R@10 on Flickr30K for Image Retrieval (IR) and Text Retrieval (TR) are reported. Dark and light grey colors highlight the top and second best results across all the tasks trained with In-domain data
['Pre-training Data', '[EMPTY]', 'Pre-training Tasks', 'Meta-Sum', 'VQA', 'IR (Flickr)', 'TR (Flickr)', 'NLVR2', 'Ref- COCO+']
Although MLM trained on text only did not absorb any image information during pre-training, we see a gain of approximately +30 on Meta-Sum over L1. Hence, we use the pre-trained weights in L2 to initialize our model for the following experiments.