A Comprehensive Survey on Cross-modal Retrieval
TABLE VI: MAP Comparison of different shallow real-valued representation learning methods on the Wiki dataset in the three-modality case.
['Query', 'Modality A', 'Modality B', 'Modality C']
Results on the Wiki in three-modality case: To evaluate the performance of cross-modal retrieval methods in a three-modality case, we test several methods on the Wiki dataset. To the best of our knowledge, there are no three or more modalities of datasets available publicly in the recent literature. The Wiki dataset contains two modalities of data: text and image. To simulate a three-modality setting, 4096-dimensional CNN ( Here 128-dim SIFT histogram, 10-dim LDA feature and 4096-dimensional CNN features are used as Modality A, Modality B and Modality C, respectively. We can see that JFSSL outperforms the other methods in three cross-modal retrieval tasks. This is mainly due to the fact that JFSSL is designed for the N-modality case, which can model the correlations between different modalities more accurately in the three-modality case. However, the other methods are designed for only the two-modality case, and they are not suitable for the three-modality case.