DramaQA: Character-Centered Video Story Understanding with Hierarchical QA
Table 2: Quantitative result about ablation study of our model on the DramaQA test set. We divided test set by difficulty level and get performance of each set. Last and second last columns respectively show the average of performance of each set and performance of overall test set, respectively. Our-{S,M,B} indicates our model without input script, behavior&emotion, and visual bounding box respectively. Our-Char.Match is our model without character match module and Our-Con.Match is our model without context matching module.
['Model', 'Diff. 1', 'Diff. 2', 'Diff. 3', 'Diff. 4', 'Overall', 'Diff. Avg.']
QA+DotProd model is designed to choose the highest score on the consine similarity between the average of question’s word embeddings and the average of corresponding candidate answer’s word embeddings. QA+MLP model concatenates average of question’s word embeddings and the average of corresponding candidate answer’s word embeddings and gets scores with softmax function after simple MLP network with 600 dimensions for input, 50 dimensions for the hidden layer, and 1 dimension for output. Our-{S,M,B} indicates our model without input script, behavior&emotion, and visual bounding box respectively. Note that three input streams are helpful to infer a correct answer. After all, most of QAs in shot level inquire visual information and most of QAs in scene level need joint understanding of visual and textual information. Our-Char. Match is our model without character match module and Our-Con. Match is our model without context matching module. Note that Con. Match module is more dominant to get correct answer than Char. Match module. Char. Match module only covers QA when different characters appear for each answer candidate, so it shows lower accuracy.