Learning Visual Reasoning Without Strong Priors
Table 1: CLEVR accuracy by baseline methods, competing methods, and our method (CBN). Methods denoted with (*) use extra supervisory information through program labels. Methods denoted with (†) use data augmentation and no pre-trained CNN.
['Model', '[BOLD] Overall', 'Count', 'Exist', 'Compare Numbers', 'Query Attribute', 'Compare Attribute']
Our model achieves a new overall state-of-the-art, outperforming humans and previous, leading models, which often use additional program supervision. We note also that our model’s pattern of performance more closely resembles that of humans than other models do. Strong performance (<1% error) in exist and query_attribute categories is perhaps explained by our model’s close resemblance to standard CNNs, which traditionally excel at these classification-type tasks. Our model also demonstrates strong performance on more complex categories such as count and compare_attribute. Comparing numbers of objects gives our model more difficulty, understandably so; this question type requires more high-level reasoning steps — querying attributes, counting, and comparing — than other question type. RNs use an MLP to carry out pairwise comparisons over each location of extracted convolutional features over an image, including LSTM-extracted question features as input to this MLP. RNs then element-wise sum over the resulting comparison vectors to form another vector from which a final classifier predicts the answer. Our approach lifts the explicitly relational aspect of this model, freeing our approach from the use of a comparison-based prior, as well as the scaling difficulties of pairwise comparisons over spatial locations.