Multi-step Reasoning via Recurrent Dual Attention for Visual Dialog
Table 5: Question-type analysis of the NDCG score achieved by different models on the val v1.0 set.
['Question Type', 'All', 'Yes/no', 'Number', 'Color', 'Others']
We further perform a question-type analysis of the NDCG scores achieved by different models. We classify questions into 4 categories: Yes/no, Number, Color, and Others. no questions, while discriminative models performed better on all the other types of questions. We hypothesize that this is due to that generative models tend to ranking short answers higher, thus is beneficial for Yes/no questions. Since Yes/no questions take a majority of all the questions (75%), the better performance of generative models on the Yes/no questions translated into an overall better performance of generative models. Aggregating the ranking results of both discriminative and generative models results in the mutual enhancement of each other, and therefore boosting the final NDCG score by a large margin. Also, we observe that the Number questions are most difficult to answer, since training a model to count is a challenging research problem.