TVR: A Large-Scale Dataset for Video-Subtitle Moment Retrieval
Table 4: XML performance on TVR val set by query types, with different input modalities.
['Model', 'IoU=0.7', 'IoU=0.7', 'IoU=0.7', 'IoU=0.7']
To show the importance of using both video and subtitle, we compare with XML variants that use only video or subtitle. Firstly, the model with both modalities as inputs has better overall performance than the single modality models, which proves that both modalities are useful for the task. Secondly, we observe that models trained on one modality do not perform well on the queries tagged by another modality. For example, the XML (video) model that uses only video performs much worse on subtitle queries compared to XML (sub) model (0.73 vs 1.87).