An Analysis of Visual Question Answering Algorithms
Table 7: Results on TDIUC-Tail for MCB model when trained on full TDIUC dataset vs when trained only on TDIUC-Tail. The normalized scores for each question-types and five different overall scores are shown here
['[EMPTY]', '[BOLD] MCB TDIUC-Full', '[BOLD] MCB TDIUC-Tail']
This shows that MCB is capable of learning to correctly predict rarer answers, but it is simply biased towards predicting more common answers to maximize overall accuracy. Using normalized accuracy disincentivizes the VQA algorithms’ reliance on the answer statistics, and for deploying a VQA system it may be useful to optimize directly for N-MPT. In this section, we present additional experimental results that were omitted from the main paper due to inadequate space. To compute these scores, the accuracy for each unique answer is calculated separately within a question-type and averaged. The results are evaluated on TDIUC-Tail, which is a subset of TDIUC that only consists of questions that have answers repeated less than 1000 times (uncommon answers). Note that the TDIUC-Tail excludes the absurd and the object presence question-types, as they do not contain any questions with uncommon answers. MCB TDIUC-Tail : MCB model trained and evaluated on TDIUC-Tail.