Deep Compositional Question Answering with Neural Module Networks
Table 3: Results on the VQA test server. NMN+LSTM is the full model shown in Figure 1, while NMN is an ablation experiment with no whole-question LSTM. The full model outperforms previous approaches, scoring particularly well on questions not involving a binary decision. Baseline numbers are as reported in previous work.
['[EMPTY]', 'test-dev Yes/No', 'test-dev Number', 'test-dev Other', 'test-dev All', 'test All']
As can be seen, we outperform the best published results on this task. A breakdown of our questions by answer type reveals that our model performs especially well on questions answered by an object, attribute, or number, but worse than a sequence baseline in the yes/no category. Inspection of training-set accuracies suggests that performance on yes/ no questions is due to overfitting. An ensemble with a sequence-only system might achieve even better results; future work within the NMN framework should focus on redesigning the measure module to reduce effects from overfitting.