DMRM: A Dual-channel Multi-hop Reasoning Model for Visual Dialog
Table 1: Performance on VisDial val v0.9 [das2017visual]. Higher the better for mean reciprocal rank (MRR) and recall@k (R@1, R@5, R@10), while lower the better for mean rank. Our proposed model outperforms all other models on MRR, R@5, and mean rank. † indicates that the model is trained by using reinforcement learning.
['Model', 'MRR', 'R@1', 'R@5', 'R@10', 'Mean']
We compare our proposed models to the state-of-the-art generative models developed in previous works. The key observations are as follows: By comparing with singe-hop approaches (LF [das2017visual] and HCAIE [ lu2017best]) , we demonstrate the validity of multi-hop reasoning, because it utilizes the abundant latent information between modalities. By comparing with singe-channel approaches (CoAtt [wu2018you] and RvA [niu2019recursive]), we come to the conclusion that dual-channel reasoning is beneficial for gaining an original understanding of the question from the dialog history and the image. By comparing with other methods and the state-of-the-art approaches (HREA [das2017visual], MN [das2017visual] and ReDAN [gan2019multi]) , our approach achieves the state-of-the-art results on some metrics that demonstrate the superiority of our model.