Pixel-BERT: Aligning Image Pixels with Text by Deep Multi-Modal Transformers
Table 2: Evaluation of Pixel-BERT with other methods on VQA.
['Model', 'test-dev', 'test-std']
We compare our approach with recent state-of-the-art approaches. When equipped with ResNeXt-152 backbone, our model Pixel-BERT achieves 74.45 on test-dev split and 74.55 on test-std split, which significantly outperforms all existing works. It worth noting that this result is even higher than the performance of UNITER (Large), which using 24-Layer Transformer as a language module and get 73.40 score on VQA test-std split. This obvious improvement demonstrates that learning visual and language attention in image’s pixel-level can benefit the visual encoder representation and enhance the visual and language embedding learning afterward. We evaluate NLVR2 on both dev and test-P split. Different from pre-training tasks and other downstream tasks, NLVR2 receives a pair of images at once. These results show that Pixel-BERT can be also adapted to other similar input formats.