Learning by Asking Questions
Table 1: CLEVR val accuracy for six budgets B. We condition the generator on the image (I) or on the image and the question type (I + qtype), vary the generator sampling temperatures τ, and use three different relevance models. We re-run the LBA pipeline for each of these settings.
['[BOLD] Generator  [ITALIC] g', '[BOLD] Relevance  [ITALIC] r', '[BOLD] Budget  [ITALIC] B 0k', '[BOLD] Budget  [ITALIC] B 70k', '[BOLD] Budget  [ITALIC] B 210k', '[BOLD] Budget  [ITALIC] B 350k', '[BOLD] Budget  [ITALIC] B 560k', '[BOLD] Budget  [ITALIC] B 630k']
Diversity is also controlled by the sampling temperature, τ, used in g. Rows 3 -5 show that a lower temperature, which gives less diverse question proposals, negatively impacts final accuracy. We can also decouple the effect of the relevance model r from the rest of our setup by replacing it with a “perfect” relevance model (the oracle) that flawlessly filters all invalid questions.