Video2Commonsense: Generating Commonsense Descriptions to Enrich Video Captioning
Table 4: Precision (p) and Recall (r) for V2C-QA for each type of question.
['[BOLD] Type', '[BOLD] Model', '[BOLD] top-1 p', '[BOLD] top-1 r', 'p', 'r', 'p', 'r']
Results MSR-VTT QA [xu2017video] acts as a good baseline since it is trained on a conventional videoQA task on the MSR-VTT videos. However this model is trained for a multiple-choice QA scheme, so we modify it with our open-ended answering module. We compare our models with when we use our encoder pretrained on the V2C caption generation task, and then finetune it on the V2C-QA task. We also train models with ground-truth factual captions as input. It can be seen that using our encoder pre-trained on the V2C task outperforms all other models. Attribute-related questions are easier to answer, while the models struggle the most for questions about intention. Captions help in questions about effects.