VisualBERT: A Simple and PerformantBaseline for Vision and Language
Figure 3: Performance of the ablation models on NLVR2. Results confirm that task-agnostic pre-training (C1) and early fusion of vision and language (C2) are essential for VisualBERT.
['[EMPTY]', 'Model', 'Dev']
For ease of computations, all these models are trained with only 36 features per image (including the full model). We also consider a baseline that always chooses the region with the highest detection confidence. We find that VisualBERT achieves a remarkably high accuracy though it is not exposed to any direct supervision for entity grounding. The grounding accuracy also seems to improve in higher layers, showing the model is less certain when synthesizing the two inputs in lower layers, but then becomes increasingly aware of how they should align.