Auto-captions on GIF: A Large-scale Video-sentence Dataset forVision-language Pre-training
Table 1: Comparison of video-sentence datasets.
['Dataset', 'Context', 'Sentence Source', '#Video', '#Sentence', '#Word', 'Vocabulary']
Note that we are continuing to crawl more GIF videos from new web pages, and thus more data will be released in the future. In current version, our Auto-captions on GIF contains 163,183 GIF videos and 164,378 sentences, and is the largest video-sentence dataset in terms of video number (163,183) and word vocabulary (31,662). Moreover, different from the most existing datasets which focus on specific fine-grained domains and require human annotations, our Auto-captions on GIF is derived from billions of web pages with massive video categories. As such, the resources can significantly benefit the generalization capability of pre-trained representation or encoder-decoder structure on downstream tasks. To sum up, Auto-captions on GIF represents the most comprehensive, diverse, and complex video-sentence dataset for video understanding, and thus can naturally facilitate the vision-language pre-training in video domain.