Visual Word2Vec (vis-w2v): Learning Visually Grounded Word Embeddings Using Abstract Scenes
Table 6: Performance on the common sense task of [34] using 4k real images with with text baseline at 68.1, initialized from w2v-wiki.
['K', '25', '50', '75', '100']
We then learn vis-w2v4k with NH=200 in the Orig/Words setting (see Sec. We observe that the performance on the validation set reduces for vis-w2v-coco4k. Observations: Comparing the improvements from real images (best AP of 73.7%) to those from abstract scenes (best AP of 74.8%), we see that that abstract visual features capture visual semantics better than real images for this task. It if often difficult to capture localized semantics in the case of real images. For instance, extracting semantic features of just the primary and secondary objects given a real image, is indeed a challenging detection problem in vision. On the other hand, abstract scene offer these fine-grained semantics features therefore making them an ideal for visually grounding word embeddings.