Factor Graph Attention
Table 1: Performance of discriminative models on VisDial v0.9. Higher is better for MRR and recall@k, while lower is better for mean rank. (*) denotes use of external knowledge.
['Model', 'MRR', 'R@1', 'R@5', 'R@10', 'Mean']
Visual question answering comparison: We first compare against a variety of baselines (see Tab. Note that almost all of the baselines (except LF, HRE and MN and SF-QIH-se-2) use attention, i.e., attention is an important element in any model. Note that our model uses the entire set of answers to predict each answer ’s score, i.e., we use p(ui|A,I,Q,C,H) This is in contrast to SF-QIH-se-2, which doesn’t use attention and models p(ui|^ui,I,Q,C,H). Because CoAtt-GAN uses a hierarchical approach, the ability to further improve the reasoning system is challenging and manual work. In contrast, our general attention mechanism allows to attend to the entire set of cues in the dataset, letting the model automatically choose the more relevant cues. We refer the readers to the appendix for analysis of utility-importance via importance score. As can be seen from Tab. We also report an ensemble of 9 models which differ only by the initial seed. We emphasize that our approach only uses VGG16. Lastly, some baselines report to use GloVe to initialize the word embeddings, while we didn’t use any pre-trained embedding weights.