Exploration on Grounded Word Embedding: Matching Words and Images with Image-Enhanced Skip-Gram Model
Table 2: Word-Image vector matching results (find image given word). P@x means if one image in the top x predicted images is correct, we think it is correct.
['Task', 'P@1', 'P@5', 'P@10']
The task is to find the most relevant image given a word. We assume that if the images and corresponding words can be matched based on their vectors, they are more likely to be in the same hyper-plane, which is the objective of our paper. From the results we can see that finding the image given a word is much more difficult than the other way around. Even P@10 can only reach 62.5 which is similar to the level of P@1 for Image to Word matching. We think this is because images are much more complicated than word embeddings, Furthermore, the compression procedure of ResNet can damage the innate structure of the image which can also hurt the matching performance. Convolutional Neural Networks classify images by focusing on one part of the image, which loses the overall information of the image.