Learning Visually-Grounded Semanticsfrom Contrastive Adversarial Samples
Table 7: Evaluation result on the fill-in-the-blank task (in percentage). The word embeddings learned by VSE-C with all classes of contrastive adversarial samples help reach a better performance than those learned by VSE++ [Faghri et al.2017].
['[BOLD] Model', '[BOLD] Noun Filling R@1', '[BOLD] Noun Filling R@10', '[BOLD] Prep. Filling R@1', '[BOLD] Prep. Filling R@10', '[BOLD] All (n. + prep.) R@1', '[BOLD] All (n. + prep.) R@10']
the performance of the proposed VSE-C on filling in both nouns and prepositions. VSE-based models outperform GloVe without visual grounding and concretely correlate word semantics with image embeddings. We found that only small gaps exist between VSE++ and VSE-C on preposition filling, which again shows the limited diversity on visual relations within the dataset.