Video Object Grounding using Semantic Roles in Language Description
Table 4: Evaluation of VOGNet in GT5 setting by training (first column) and testing (top row) on SVSQ, TEMP, SPAT respectively
['[EMPTY]', 'SVSQ Acc', 'SVSQ SAcc', 'TEMP Acc', 'TEMP SAcc', 'SPAT Acc', 'SPAT SAcc']
However, the reverse is not true i.e. models trained on SVSQ fail miserably in SPAT and TEMP (accuracy is <3%). This suggests that both TEMP and SPAT moderately counter the bias caused by having a single object instance in a video. Interestingly, while VOGNet trained on TEMP doesn’t perform well on SPAT (performance is worse than VidGrnd trained on SPAT), when VOGNet is trained on SPAT and tested on TEMP it significantly outperforms VidGrnd trained in TEMP. This asymmetry is possibly because the multi-modal transformer is applied to individual frames.