MUREL: Multimodal Relational Reasoning for Visual Question Answering
Table 1: Comparing MuRel to Attention. Comparison of the MuRel strategy against a strong Attention-based model on the VQA 2.0 val, VQA-CP v2 and TDIUC datasets. Both models have an equivalent number of parameters (∼60 million) and are trained on the same features following the same experimental setup.
['Model', 'VQA 2.0', 'VQA CP v2', 'TDIUC']
The goal of this experiments is to compare our approach with strong baselines for real VQA in controlled conditions. In addition to using the same bottom-up features, which are crucial for fair comparisons, we also dimension the attention-based baseline to have an equivalent amount of learned parameters than MuRel (∼60 millions including those from the GRU encoder). Also, we train it following the same experimental setup to insure competitiveness. MuRel reaches a higher accuracy on the three datasets. We report a significant gain of +1.70 on VQA 2.0 and +1.50 on VQA CP v2. Not only these results validate the ability of MuRel to better model interactions between the question and the image, but also to generalize when the distribution of the answers per question are completely different between the training and validation set as in VQA CP v2. A gain of +1.24 on TDIUC demonstrates the richer modeling capacity of MuRel in a fine-grained context of 12 well delimited question types.