Aligning Multilingual Word Embeddings for Cross-Modal Retrieval Task
Table 1: Image-caption ranking results for English (Multi30k)
['[EMPTY]', 'Image to Text R@1', 'Image to Text R@5', 'Image to Text R@10', 'Image to Text Mr', 'Text to Image R@1', 'Text to Image R@5', 'Text to Image R@10', 'Text to Image Mr', 'Alignment']
we show the results for English and German captions. For English captions, we see 21.28% improvement on average compared to Kiros et al. (2014). There is a 1.8% boost on average compared to Mono due to more training data and multilingual text encoder. AME performs better than FME model on both symmetric and asymmetric modes, which shows the advantage of finetuning word embeddings during training. We have 25.26% boost on average compared to Kiros et al. (2014) in asymmetric mode.