Modality-Balanced Models for Visual Dialogue
Table 5: Performance comparison between our models and other models on the test-standard dataset of VisDial v1.0. We run two ensemble models each from 6 image-only models and 6 consensus dropout fusion models.
['[EMPTY]', 'Models', 'NDCG', 'MRR', 'R@1', 'R@5', 'R@10', 'Mean']
For the evaluation on the test-standard dataset of VisDial v1.0, we try 6 image-only model ensemble and 6 consensus dropout fusion model ensemble. Specifically, our image-only model shows much higher NDCG score (60.16). On the other hand, our consensus dropout fusion model shows more balanced results over all metrics while still outperforming on most evaluation metrics (NDCG, MRR, R@1, and R@5). Compared to results of the Visual Dialog challenge 2019, our models also show strong results. Although ReDAN+ [gan2019multi] and MReaL–BDAI show higher NDCG scores, our consensus dropout fusion model shows more balanced results over metrics while still having a competitive NDCG score compared to DAN [kang2019dual], with rank 3 based on NDCG metric and high balance rank based on metric average.