From Visual Attributes to Adjectivesthrough Decompositional Distributional Semantics
Table 6: Percentage hit@k noun retrieval scores.
['[EMPTY]', 'DiRO', 'Dec', 'DiRA']
Interestingly, accounting for the phrasal nature of visual information leads to substantial performance improvement in object recognition through zero-shot learning (i.e., tagging images with the depicted nouns) as well. Note that DiRO represents the label retrieval technique that has been standardly used in conjunction with zero-shot learning for objects : The cross-modal function is trained on images annotated with nouns that denote the objects they depict, and it is then used for noun label retrieval of unseen objects through a nearest neighbor search of the mapped image representation (the DiRA column shows that zero-shot noun retrieval using the mapping function trained on adjectives works very poorly). Dec decomposes instead the mapped image representation into two vectors denoting adjective and noun semantics, respectively, and uses the latter to perform the nearest neighbor search for a noun label.