Improving zero-shot learning by mitigating the hubness problem
Table 2: Percentage label and image retrieval accuracy.
['Train', 'Chance', 'GCV NN', 'GCV NNnrm', 'GCV GC']
We consider both the usual image labeling setting (Vision→Language) and the image retrieval setting (Language→Vision). For the Vision→Language task, we use as pivot set the 100K test images (1,000 labels x 100 images/label) and an additional randomly chosen 100K images. The search space is the entire label set of 5,000 words. For Language→Vision, we use as pivot set the entire word list (5,000) and the target set is the entire set of images (500,000). The objects depicted in the images form a set of 5,000 distinct elements, therefore, for the word cat, for example, returning any of the 100 cat images is correct. Chance accuracy in both settings is thus at 1/5,000. We observe that, differently from the translation case, correcting by normalizing the cosine scores of the elements in the target domain (NNnrm) leads to poorer results than no correction. On the other hand, the GC method is consistent across domains, and it improves significantly on the standard NN method in both settings. Note that, while there are differences between the setups, Frome et al.