Joint Image Captioning and Question Answering
Table 3: Evaluation of the effectiveness of semantic connection modeling in test-standard split. Accuracies in percentage (%) are reported.
['[EMPTY]', 'Test-standard All', 'Test-standard Yes/No', 'Test-standard Num', 'Test-standard Other']
Ablation study on the semantic connection modeling In this section , we quantitatively analyze the effectiveness of incorporating captions to model the relationship besides the advantage of providing additional knowledge. The without relationship modeling methods is that we only use the caption features as inputs but don’t involve them in the visual attention parts (i.e. we don’t compute Acv). As demonstrated in Table. 0.5% improvements are gained from adopting caption features to model the attention of images feature V in both validation and test-standard splits. We use w SC to indicate with semantic connection modeling modeling and w/o SC to indicate without.