A Comprehensive Survey on Cross-modal Retrieval
TABLE VII: MAP comparison of different deep real-valued representation learning methods on the Flickr30K dataset.
['Methods', 'Sentence retrieval R@1', 'Sentence retrieval R@5', 'Sentence retrieval R@10', 'Sentence retrieval Med r', 'Image retrieval R@1', 'Image retrieval R@5', 'Image retrieval R@10', 'Image retrieval Med r']
End-to-end DCCA and Deep Fragment perform better than other methods. The reason is that Deep Fragment breaks an image into objects and a sentence into dependency tree relations, and maximises the explicit alignment between the image fragments and text fragments. For End-to-end DCCA, the used TF-IDF based text features and CNN based visual features capture global properties of the two modalities respectively. The alignment of the fragments in image and text is implicitly considered by the CCA correlation objective.