Deep Captioning with Multimodal Recurrent Neural Networks (m-RNN)
Table 3: Results of R@K and median rank (Med r) for Flickr8K dataset. “-AlexNet” denotes the image representation based on AlexNet extracted from the whole image frame. “-RCNN” denotes the image representation extracted from possible objects detected by the RCNN algorithm.
['[EMPTY]', 'Sentence Retrival (Image to Text) R@1', 'Sentence Retrival (Image to Text) R@5', 'Sentence Retrival (Image to Text) R@10', 'Sentence Retrival (Image to Text) Med r', 'Image Retrival (Text to Image) R@1', 'Image Retrival (Text to Image) R@5', 'Image Retrival (Text to Image) R@10', 'Image Retrival (Text to Image) Med r']
This dataset was widely used as a benchmark dataset for image and sentence retrieval. We compare our model with several state-of-the-art methods: SDT-RNN (Socher et al. (Frome et al. , DeepFE ( Karpathy et al. with various image representations. Our model outperforms these methods by a large margin when using the same image representation (e.g. AlexNet). “-avg-RCNN” denotes methods with features of the average CNN activation of all objects above a detection confidence threshold. DeepFE-RCNN Karpathy et al. The results show that using these features improves the performance. Even without the help from the object detection methods, however, our method performs better than these methods in almost all the evaluation metrics. We will develop our framework using better image features based on object detection in the future work.