DialGraph: Sparse Graph Learning Networks for Visual Dialog
Table 1: Comparison with the baseline models on the VisDial v1.0 validation splits. All models in this table are based on the discriminative decoder.
['Method', 'NDCG', 'MRR', 'R@1', 'R@5', 'R@10', 'Mean', 'Sparsity']
4.2.1 Comparison with Baselines. We compare SGLNs to the baseline models to demonstrate the effectiveness of our method. We define two models as baselines: Dense, and Sparse-hard. The Dense model utilizes a softmax attention mechanism, which results in the fully-connected graph. Contrary to the Dense model, the Sparse-hard model picks exactly one element among the dialog history by applying the Gumbel-Softmax to the whole dialog history. Note that the structural supervision is provided in the Sparse-hard model. The SGLNs achieve better performance than the baseline models on the NDCG metric, maintaining competitive performance on the ground-truth dependent metrics (i.e., MRR, R@k, and Mean rank). We also observe that the Dense model, which overly exploits the dialog history, shows the best performance on the ground-truth dependent metrics. We argue that the Dense model mainly focuses on finding the single ground-truth answer with a rich set of dialog history, with the cost of sacrificing the ability to provide ‘flexible’ answers (i.e., NDCG). Similarly, the NDCG performance for the Sparse-hard model tends to increase as the sparsity increased.