Violin: A Large-Scale Dataset for Video-and-Language Inference
Table 4: Accuracy in human evaluation on test set over different input sources.
['Source', 'Test Accuracy (%)']
As expected, humans achieve the best performance when provided with both video and subtitles (85.20) Without context (video and subtitles), humans only achieve 51.38% accuracy. Interestingly, we find that adding video brings in more gain than adding subtitles, showing the importance of visual information in Violin task.