Show, Recall, and Tell: Image Captioning with Recall Mechanism
Table 3: Performance of text-retrieval model on MSCOCO Karpathy validation set.
['[EMPTY]', 'R@1', 'R@5', 'R@10', 'Mean r']
The performance of text-retrieval model can directly affect the relevance between the recalled words and image. We assume that our text-retrieval model does not achieve state-of-the-art as in [lee2018stacked], but it is well qualified to retrieve sufficient and relevant words for images. Our corpus is collected from all the captions of MSCOCO Karpathy train splits. For determining an appropriate number of captions to be retrieved, we respectively retrieve top 1, 5 and 15 related captions for each image, and test the performance on cross-entropy loss. is a better choice than 1 or 15. It is necessary to emphasize that we avoid retrieving the ground truth caption for images, and all the retrieved captions only come from the train splits of MSCOCO Karpathy. In the following experiments, top 5 captions retrieved are used to construct recalled words for each image.