Graph-Structured Representations for Visual Question Answering
Table 1: Results on the test set of the “balanced” dataset [31] (in percents , using balanced versions of both training and test sets). Numbered rows report accuracy over pairs of complementary scenes for ablated versions of our method.
['(1) Question: no parsing (graph with previous/next edges)', '37.9']
They all use an ensemble of models exploiting either an LSTM for processing the question, or an elaborate set of hand-designed rules to identify two objects as the focus of the question. The visual features in the three models are respectively empty (blind model), global (scene-wide), or focused on the two objects identified from the question. These models are specifically designed for binary questions, whereas ours is generally applicable. Differences in performance are mostly visible in the “pairs” setting, which we believe is more reliable as it discards ambiguous test questions on which human annotators disagreed.