Being Negative but Constructively: Lessons Learnt from Creating Better Visual Question Answering Datasets
Table 10: Test accuracy (%) on VQA2−-2017val, which contains 134,813 triplets.
['Method', 'IoU', 'QoU', 'IoU +QoU']
Besides, we also perform a more in-depth experiment on VQA2, removing triplets with Yes /No as the target. , we see that the overall performance for each model decreases as the dataset becomes more challenging on average. Specifically, the model that observes question and answer on VQA2− performs much worse than that on VQA2 (37.2% vs. 48.1%).