Yin and Yang: Balancing and Answering Binary Visual Questions
Table 2: Evaluation on balanced test set. All accuracies are calculated using the VQA [2] evaluation metric.
['[EMPTY]', 'Training set Unbalanced', 'Training set Balanced']
We also evaluate all models trained on the train splits of both the unbalanced and balanced datasets, by testing on the balanced test set. Training on balanced is better. both language+vision models trained on balanced data perform better than the models trained on unbalanced data. This may be because the models trained on balanced data have to learn to extract visual information to answer the question correctly, since they are no longer able to exploit language biases in the training set. Where as models trained on the unbalanced set are blindsided into learning strong language priors, which are then not available at test time.