MovieQA: Understanding Stories in Movies through Question-Answering
Table 6: Accuracy for Video-based QA and late fusion of Subtitle and Video scores.
['Method', 'Video', 'Subtitle', 'Video+Subtitle']
We see that learning to answer questions using video is still a hard problem with performance close to random. As visual information alone is insufficient, we also perform and experiment combining video and dialog (subtitles) through late fusion. We train the SSCB model with the visual-text embedding for subtitles and see that it yields poor performance (22.3%) compared to the fusion of all text features (27.7%). For the memory network, we answer subtitles as before using Word2Vec.