Fusion of Detected Objects in Text for Visual Question Answering
Table 2: Experimental results on VCR, incorporating those reported by Zellers et al. (2019). The proposed B2T2 model and the B2T2 ensemble outperform published and unpublished/undocumented results found on the VCR leaderboard at visualcommonsense.com/leaderboard as of May 22, 2019.
['Model', 'Model', '[ITALIC] Q→ [ITALIC] A Val', '[ITALIC] Q→ [ITALIC] A Test', '[ITALIC] QA→ [ITALIC] R Val', '[ITALIC] QA→ [ITALIC] R Test', '[ITALIC] Q→ [ITALIC] AR Val', '[ITALIC] Q→ [ITALIC] AR Test']
Our Dual Encoder model worked surprisingly well compared to Zellers et al. We also evaluate a Text-Only baseline, which is similar to the Dual Encoder model but ignores the image. The ensemble of B2T2 models, pretrained on Conceptual Captions, obtained absolute accuracy improvements of 8.9%, 9.8% and 13.1% compared to the published R2C baseline for the Q→A, QA→R, and Q→AR tasks respectively. At the time of this writing (May 22, 2019), both our single B2T2 and ensemble B2T2 models outperform all other systems in the VCR leaderboard.