TVQA+: Spatio-Temporal Grounding for Video Question Answering
Table 4: Ablation study of STAGE on TVQA+ val set. Each row adds an extra component to the row above it.
['Model', 'QA', 'Grd.', 'Temp.', 'ASA']
Given the full STAGE model defined in Sec. We further replace the CNN encoders with RNN encoders, and remove the aligned fusion from the backbone model. This baseline model uses RNN to encode input sequences and interacts QA pairs with subtitles and videos separately. The final confidence score is the sum of the confidence scores from the two modalities. In the backbone model, we align subtitles with video frames from the start, fusing their representation conditioned on the input QA pair, as in Fig. We believe this aligned fusion is essential for improving QA performance, as the latter part of STAGE has a joint understanding of both video and subtitles. With both changes, our backbone model obtains 68.31% on QA Acc., significantly higher than the baseline’s 65.79%.