A Critical Analysis of What is Learned in Visually Grounded Neural Syntax Acquisition
Table 3: Pearson correlation coefficient of concreteness estimates between our 1,sWS,cME variant and existing concreteness estimates, including reproduced estimates derived from VG-NSL by Shi2019:vgsnl.
['Model', '1,sWS,cME']
We hypothesize that the core visual aspect learned, which is captured even in the 1d setting, is noun concreteness. We observe that the concreteness estimated by our model correlates with nouns that are relatively easier to ground visually in MSCOCO images. For example, nouns like “giraffe” and “elephant” are considered most concrete. These nouns are relatively frequent in MSCOCO (e.g., “elephant” appears 4,633 times in the training captions) and also have a low variance in their appearances. On the other hand, nouns with high variance in images (e.g., “traveller”) or abstract nouns (e.g., “chart”, “spot”) are estimated to have low concreteness.