Visual Coreference Resolution in Visual Dialog using Neural Module Networks
Table 3: Retrieval performance on the validation set of VisDial v0.9 [13] (discriminative models) using VGG [42] features (except last row). Higher the better for mean reciprocal rank (MRR) and recall@k (R@1, R@5, R@10), while lower the better for mean rank. Our CorefNMN model outperforms all other models across all metrics.
['[BOLD] Model', '[BOLD] MRR', '[BOLD] R@1', '[BOLD] R@5', '[BOLD] R@10', '[BOLD] Mean']
Results. We summarize our observations from Tab. below: (a) Our CorefNMN outperforms all other approaches across all the metrics, highlighting the importance of explicitly resolving coreferences for visual dialog. Specifically, our R@k (k=1,2,5) is at least 1 point higher than the best prior work (AMEM+SEQ-QI), and almost 2 points higher than NMN. (b) Removing memory augmentation (CorefNMN∖Mem) hurts performance uniformly over all metrics, as the model is unable to peek into history to decide when to resolve coreferences via the Refer module. Modules on captions seems to have varied effect on the full model, with decrease in R@1, but marginal increase or no effect in other metrics. (c) Fig.