Video Object Grounding using Semantic Roles in Language Description
Table 7: Ablative study comparing gains from Multi-Modal Transformer (MTx) and Object Transformer (OTx) and Relative Position Encoding (RPE). L: Number of Layers, H: Number of Heads in the Transformer. Note that VOGNet = ImgGrnd +MTx(1L,3H) +OTx(1L,3H) + RPE
['SPAT', 'Acc', 'VAcc', 'Cons', 'SAcc']
Ablation Study: We observe: ( i) self-attention via object is an effective way to encode object relations across frames (ii) multi-modal transformer applied on individual frames gives modest gains but falls short of object transformer due to lack of temporal information (iii) relative position encoding (RPE) boosts strict accuracy for both transformers (iv) object transformer with 3 layers and 6 heads performs worse than using a single multi-modal transformer i.e. adding more layers and attention heads to object transformer is not enough (v) using both object and multi-modal transformers with more layers and more heads gives the best performing model.