VirTex: Learning Visual Representationsfrom Textual Annotations
(a) Pretraining tasks using captions.
['[BOLD] Pretraining Task', '[BOLD] VOC07', '[BOLD] IN-1k']
Bicaptioning outperforms forward captioning; we hypothesize that bidirectional modeling gives a denser supervisory signal from each caption, improving the visual features. Bicaptioning and forward captioning both outperform Token Classification, demonstrating that learning to model the sequential structure of language also results in better visual features. All three models trained with language outperform the two models trained with object labels, showing the benefit of learning from the semantically denser language.