Supervised Learning of Universal Sentence Representations fromNatural Language Inference Data
Table 5: COCO retrieval results. SkipThought is trained either using 82k training samples with VGG19 features, or with 113k samples and ResNet-101 features (our setting). We report the average results on 5 splits of 1k test images.
['[BOLD] Model', '[BOLD] Model', '[BOLD] Caption Retrieval  [BOLD] R@1', '[BOLD] Caption Retrieval  [BOLD] R@5', '[BOLD] Caption Retrieval  [BOLD] R@10', '[BOLD] Caption Retrieval  [BOLD] Med r', '[BOLD] Image Retrieval  [BOLD] R@1', '[BOLD] Image Retrieval  [BOLD] R@5', '[BOLD] Image Retrieval  [BOLD] R@10', '[BOLD] Image Retrieval  [BOLD] Med r']
, we report results for the COCO image-caption retrieval task. We report the mean recalls of 5 random splits of 1K test images. When trained with ResNet features and 30k more training data, the SkipThought vectors perform significantly better than the original setting, going from 33.8 to 37.9 for caption retrieval R@1, and from 25.9 to 30.6 on image retrieval R@1. Our approach pushes the results even further, from 37.9 to 42.4 on caption retrieval, and 30.6 to 33.2 on image retrieval. This supports the claim that pre-trained representations such as ResNet image features and our sentence embeddings can achieve competitive results compared to features learned directly on the objective task.