Lessons learned in multilingual grounded language learning
Table 2: English Image-to-text (I→T) and text-to-image (T→I) retrieval results on the comparable part of Multi30K, measured by Recall at 1, 5 at 10. Typewriter font shows performance of two sets of symmetric and asymmetric models from gella2017image.
['[EMPTY]', '[EMPTY]', 'I→T R@1', 'I→T R@5', 'I→T R@10', 'T→I R@1', 'T→I R@5', 'T→I R@10']
Overall our scores are higher than \newcitegella2017image, which is most likely due to the different image features (\newcitefaghri2017vse++ also report a large performance gain when they use the ResNet instead of the VGG image features). Nevertheless, our results show a similar trend to the symmetric cosine similarity models from \newcitegella2017image: our best results are achieved with bilingual joint training with the added c2c objective. Their models trained with an asymmetric similarity measure show a different trend: the monolingual model is stronger than the bilingual model, and the c2c loss provides no clear improvement.