Learning to Answer Questions From Image Using Convolutional Neural Network
Table 1: Image QA performances on DAQUAR-All.
['[EMPTY]', 'Accuracy', 'WUPS @0.9', 'WUPS @0.0']
For DAQUAR-All and DAQUAR-Reduced datasets with multiple words as the answer to the question, we treat the answer comprising multiple words as an individual class for training and testing. For the DAQUAR-All dataset, we evaluate the performances of different image QA models on the full set (“multiple words”). The answer to the image and question pair may be a single word or multiple words. Our proposed CNN model significantly outperforms the multi-world approach and Neural-Image-QA in terms of accuracy, WUPS@0.0, and WUPS@0.9. Specifically, our proposed CNN model achieves over 20% improvement compared to Neural-Image-QA in terms of accuracy on both “multiple words” and “single word”. The results, shown in Table. The image component is thus of great help to the image QA task. One can also see that the performances on “multiple words” are generally inferior to those on “single word”.