Co-attending Free-form Regions and Detections with Multi-modal Multiplicative Feature Embedding for Visual Question Answering
Table 1: Evaluation results by our proposed method and compared methods on the VQA dataset.
['[EMPTY]', 'Test-dev Open-Ended', 'Test-dev Open-Ended', 'Test-dev Open-Ended', 'Test-dev Open-Ended', 'Test-dev MC', 'Test-std Open-Ended', 'Test-std Open-Ended', 'Test-std Open-Ended', 'Test-std Open-Ended', 'Test-std MC']
We only compare results of the single models since most approaches do not adopt the model ensemble strategy. Specifically, in the question types of number and other, our proposed approach brings 2.49% and 2.12% improvements on the test-std set. We also compare our approach with a baseline model Dual-MLB, which consists of two attention branches based on the MLB tensor fusion module. The baseline model fuses the question and free-form image region embedding by MLB in one branch, and fuses question and image detection embedding by MLB in another branch. The baseline Dual-MLB also outperforms MLB by 0.23% on the test-dev set, which shows that our improvements are not only from the integration of the two attention mechanisms, but are also caused by effective joint feature embedding of question, image, and detection features.