Are we asking the right questions in MovieQA?
Table 5: Performance of two QA only models on TVQA dataset - 1) WikiWord embedding model 2) TVQA baseline model [lei2018tvqa] proposed in the paper. For both the models we experiment with word embeddings trained from different data and observe that the performance doesn’t change.
['Model', 'Word embedding', 'Val accuracy']
It is worth exploring the performance of our WikiWord embedding more generally on other datasets. TVQA [lei2018tvqa] is a recent video QA dataset collected from 6 TV series. In contrast to MovieQA, the Mechanical Turkers actually watched the videos (and also read the aligned subtitles) while generating the QAs for TVQA. Since there are no equivalent to movie plots for the TV series, we perform experiments by training word embedding with the subtitles for TVQA dataset. This shows that TVQA dataset better controls for the biases that MovieQA has.