Learning Compact Reward for Image Captioning
Table 4: Results of rewriting caption from the located position by rAIRL on MS COCO standard split. Beside each score we report its improvement relative to rewriting from a random position.
['Source Caption', 'BLEU1 score', 'BLEU1 Δ', 'BLEU2 score', 'BLEU2 Δ', 'BLEU3 score', 'BLEU3 Δ', 'BLEU4 score', 'BLEU4 Δ', 'ROUGE_L score', 'ROUGE_L Δ', 'CIDEr score', 'CIDEr Δ', 'SPICE score', 'SPICE Δ']
S5SS2SSS0Px3 Diagnose and improve captions. Since the proposed rAIRL learns a word-wise reward, it’s also applicable to diagnose a given caption by finding the wrong word (e.g., the word whose reward decreases sharply compared with that of the previous word) and rewriting the caption to improve its quality. Therefore, we choose to rewrite a given caption (source caption) from the word whose reward has a decrease rate larger than 50%. However, we found that even rewriting the source caption from a random position using rAIRL can also improve the evaluation scores. Thus, rewriting from a random position is selected as the baseline to compare with rewriting from the located position. Beside each score we report its improvement relative to rewriting from a random position, whose values are mostly positive. This demonstrates that the proposed rAIRL can diagnose the caption at a word level, and further improves the caption quality by rewriting from the located position.