The Promise of Premise: Harnessing Question Premisesin Visual Question Answering
Table 6: Performance of DeeperLSTM (Antol et al., 2015) on Compositional VQA test split with different augmentations.
['Data Ablation', 'Overall', 'Other', 'Number', 'Yes/No']
We evaluate the Deeper LSTM model of Lu et al. All which includes the entire set of premise questions and Top-1k-A which includes only questions with answers in the top 1000 most common VQA answers. We find minor improvement of 0.34% on the standard split under Top-1k-A premise question augmentation. On the compositional split, we observe a 1.16% gain with Top-1k-A augmentation over no augmentation. In this setting, explicitly reasoning about objects and attributes seen in the questions seems to help the model disentangle objects from their common characteristics. While all settings show some improvements over the standard training set, we find the largest increase with the Top1k-A setting. By restricting the additional question to those having answers in the top-1000 most commonly occurring answers from the standard VQA set, the added data does not significantly shift the types of answers the model learns are likely. Some examples where the augmented DeeperLSTM model performs better than a non-augmented model are shown in \figreffig:compositionality.