Incorporating External Knowledge to Answer Open-Domain Visual Questions with Dynamic Memory Networks
Table 2: Accuracy on our generated open-domain dataset.
['Methods', 'Accuracy']
Since most of the alternative methods do not provide the results in the open-domain scenario, we make comprehensive comparison with our ablative models. As expected, we observe that a significant improvement (12.7%) of our full KDMN model over the KDMN-NoKG model, where 6.8% attributes to the involvement of external knowledge and 5.9% attributes to the usage of memory network. Examples in Fig. It is difficult or even impossible for a system to answer the open domain question when comprehensive reasoning beyond image content is required , e.g., the background knowledge for prices of stuff is essential for a machine when inferring the expensive ones. The larger performance improvement on open-domain dataset supports our belief that background knowledge is essential to answer general visual questions. Note that the performance can be further improved if the technique of ensemble is allowed. We fused the results of several KDMN models which are trained from different initializations. Experiments demonstrate that we can further obtain an improvement about 3.1%.