Co-attending Free-form Regions and Detections with Multi-modal Multiplicative Feature Embedding for Visual Question Answering
Table 2: Ablation study on the VQA dataset, where “*” denotes our model’s design.
['[BOLD] Method', '[BOLD] Validation']
In this section, we conduct ablation experiments to study effectiveness of individual component designs in our model. Following other compared approaches, the test set is not used in the study due to online submission restrictions. Specifically, we compare different multi-modal feature embedding designs and investigate the roles of the two attention mechanisms. Element-wise multiplication (denoted as MFA-MUL) in Eq. For fusing outputs hr and hd from the two attention branches, element-wise addition in Eq. approach is also used for comparison. Results show that our models with a single attention branch outperform all compared detection-based and region-based baselines. Finally, we compare our final model Dual-MFA with the baseline model Dual-MLB. Our Dual-MFA model benefits from its effective multi-modal multiplicative feature embedding scheme and achieves an improvement of 0.75%.