Compositional Generalization in Image Captioning
Table 3: Detailed Recall@5 scores for different categories of held out pairs. The scores are averaged over the set of scores for pairs from the respective category. rr stands for re-ranking after decoding. Color and size adjectives are split into Animate or Inanimate objects; Verbs are split into Transitive and Intransitive verbs.
['[EMPTY]', 'Color A', 'Color I', 'Size A', 'Size I', 'Verb T', 'Verb I']
We observe that all models are generally best at describing colors, especially of inanimate objects; they nearly never correctly describe held out size modifiers; and for held out noun–verb pairs, performance is slightly better for transitive verbs.