Visual Question Answering for Cultural Heritage
Table 1: Question classifier: accuracy of our question classifier on questions from both the OK-VQA and VQA v2 datasets and from Artpedia.
['[EMPTY]', 'OK-VQA/VQA v2', 'Artpedia']
Question Classifier We train the question classifier module with questions of both the OK-VQA and VQA v2 datasets. We take from VQA v2 a number of visual questions equal to the number of questions that require external knowledge from OK-VQA. The obtained dataset is then split into train and test sets. The question classifier is supposed to understand from the structure of the question whether the answer concerns the visual content or not. This is a generic classifier, agnostic from the domain of the task. In fact, VQA v2 and OK-VQA contain generic images, while we are interested in applications in the cultural heritage domain. Since this dataset does not contain questions but only images and descriptions, we took 30 images from this dataset and annotated them with a variable number of both visual and contextual questions (from 3 to 5 for both categories). The accuracy of our question classifier module is shown in Tab. We can observe that it is able to predict the type of the question correctly in most cases.