VirTex: Learning Visual Representationsfrom Textual Annotations
Table 2: Ablations: Transformer Size. Comparison across varying Transformer width (hidden size H) and depth (layers L). All models use ResNet-50 as the visual backbone. We show that larger transformers improve downstream linear classification performance – both, wider (left) and deeper (right).
['[BOLD] Depth  [ITALIC] L', '[BOLD] Width  [ITALIC] H', '[BOLD] VOC07', '[BOLD] IN-1k']
Increasing width consistently improves downstream classification performance. Similar to width, increasing depth consistently improves downstream classification performance. However, based on these empirical observations, we think that even larger widths and depths may continue to improve performance. (c) Object Detection on PASCAL VOC In all these fine-tuning experiments, we use the VirTex model with ResNet-50 visual backbone, and textual head with depth L=1, width H=2048