VizWiz Grand Challenge: Answering Visual Questions from Blind People
Table 4: Accuracy of nine VQA algorithms for visual questions that lead to different answer types.
['[EMPTY]', 'Yes/No', 'Number', 'Unans', 'Other']
We further enrich our analysis by evaluating the nine algorithms for visual questions that lead to different answer types (their frequencies in VizWiz are shown in parentheses): “yes/no” (4.80%), “number” (1.69%), “other” (58.91%), and “unanswerable” (34.6%). Overall, we observe performance gains by fine-tuning algorithms (rows 4–6) and training from scratch (rows 7–9), with the greatest gains for “unanswerable” visual questions and smallest gains for “number” and “other” visual questions. Exemplar failures include when asking for text to be read (e.g., captchas, cooking directions) and things to be described (e.g., clothes).