Learning Compact Reward for Image Captioning
Table 2: Correlation between the reward differences and semantic differences by replacing a given word with a similar word (RP S) and a distinct word (RP D), respectively.
['Method', 'Standard Split RP S', 'Standard Split RP D', 'Robust Split RP S', 'Robust Split RP D']
Compactness means that the reward values should be close for similar words and different for distinct words. For example, kid can also be referred to as little boy or little girl, and thus their reward values should be close to each other in the discriminator. To see the correlation between the reward differences and semantic differences, we replace a given word wt in the generated caption with a similar word wsimilar and a distinct word wdistinct, respectively. A sentence is discarded if no word can be replaced. The words within the same class are considered to be similar (such as bike and bicycle), and the words that belong to difference classes are distinct (such as man and bike). For reward-learning methods, the reward difference is the variation of the learned reward. Higher correlation indicates better compactness. RL serves as a baseline in that the handcrafted reward SPICE compares n-gram overlapping without considering the semantic difference. The reward differences of rAIRL correlate the best with the semantic differences for both similar words and distinct words, proving the compactness of the learned reward. It’s also noted that due to the reward ambiguity problem, the reward differences of GAN poorly correlate with the semantic differences, especially for similar words. Driven by the learned compact reward function, the top-5 captions of an image have the same format whilst some words are replaced with their synonyms. For example, the model uses blue and white bus, city bus and public transit bus to describe the bus in the second picture, while the format of its generated sentence remains the same.