Using sparse semantic embeddings learned from multimodal text and image data to model human conceptual knowledge
Table 4: Average RSA results (Spearman’s ρ) for all sparse (S) and dense (D) models.
['[EMPTY]', '[BOLD] GloVe', '[BOLD] Word2Vec', '[BOLD] CNN-Max', '[BOLD] CNN-Mean', '[BOLD] CNN-Max + GloVe', '[BOLD] CNN-Mean + GloVe', '[BOLD] CNN-Max + Word2Vec', '[BOLD] CNN-Mean + Word2Vec']
The results show that sparse models give the closest representation to both fMRI and MEG data, with the multimodal sparse word2vec+CNN-Mean model best fitting the fMRI data, and the sparse GloVe model best fitting the MEG data. These results indicate that semantic model sparsity is an important property reflected in neurocognitive semantic representations.