Reasoning Visual Dialogs with Structural and Partial Observations
Table 1: Quantitative evaluation of discriminative methods on val split of VisDial v0.9 [9]. Our model outperforms most competitors. See subsection 4.1 for more details.
['Methods', 'MRR ↑', 'R@1 ↑', 'R@5 ↑', 'R@10 ↑', 'Mean ↓']
Quantitative Results Our model consistently outperforms most approaches, highlighting the importance of understanding the dependencies in visual dialog. Specifically, our R@k (k = 1, 5, 10) is at least 0.4 point higher than SF. Our method only performs slightly worse than SCA, which adopts adversarial learning techniques. Qualitative Results: We summarize three key observations: (i) We compare our machine selected answer with human answer and show that our model is capable of selecting meaningful yet different answers compared with the ground-truth answer. (ii) We present our inferred dialog structure according to the edge weight between each pair of nodes. We show that the edge weight is relatively high when the correlation between the node pairs is strong. (iii) As seen, the suggested model successfully captures the relations in dialog and attend to dialog fragments which are relevant to current question.