Learning by Asking Questions
Table 4: Accuracy on CLEVR validation data during training as a function of the number of images used.
['[BOLD] Number of images', '[BOLD] Budget  [ITALIC] B 0k', '[BOLD] Budget  [ITALIC] B 70k', '[BOLD] Budget  [ITALIC] B 140k', '[BOLD] Budget  [ITALIC] B 210k', '[BOLD] Budget  [ITALIC] B 350k', '[BOLD] Budget  [ITALIC] B 560k', '[BOLD] Budget  [ITALIC] B 630k']
Our experiments in Section 5.4 (main paper) show that the LBA setting can be more sample-efficient than a i.i.d. data-collection strategy, i.e., a smaller budget of oracle queries leads to a higher accuracy. We investigate what is more important for learning: the number of images or the number of questions. Specifically, we vary the size of the image set Dtrain used for the LBA setting. Using 35k images, our model achieves the same accuracy as a model trained on the full CLEVR training set (which has 70k images).