Visual Coreference Resolution in Visual Dialog using Neural Module Networks
Table 2: Answer accuracy on MNIST Dialog dataset. Higher the better. Our CorefNMN outperforms all other models with a near perfect accuracy on test set.
['[BOLD] Model', '[BOLD] Acc.']
Results. Tab. The following are the key observations: (a) The text-only Q (36.6%) and image- only I (20.2%) do not perform well, perhaps as expected as MNIST Dialog needs resolving strong coreferences to arrive at the correct answer. Interestingly, Q outperforms NMN by around 13% (both use question and image, but not history), possibly due to the explicit reasoning nature of NMN prohibiting it from capturing the statistic dataset priors. (b) Our CorefNMN outperforms all other models with near perfect accuracy of 99.3%. Examining the failure cases reveals that most of the mistakes made by CorefNMN was due to misclassifying qualitatively hard examples from the original MNIST dataset. (c) Factoring the sequential nature of the dialog additionally in the model is beneficial, as indicated by the 10.6% improvement in CorefNMN, and 7.2% in AMEM. Intuitively, phrases with multiple potential referents, more often than not, refer to the most recent referent, as seen in Fig. Fig.