Using sparse semantic embeddings learned from multimodal text and image data to model human conceptual knowledge
Table 2: Average cross-validation F1 ×100 scores for each model. The blue highlighting indicates the model that scores the highest on each property class.
['[BOLD] Model', '[BOLD] Encyclopedic', '[BOLD] Functional', '[BOLD] Taxonomic', '[BOLD] Visual', '[BOLD] Other Perceptual', '[BOLD] Overall']
Since the dense and sparse models trained on the same source data (text, images, or text+images) encode similar information, they perform similarly on the task of predicting human semantic property knowledge. the top three models are all sparse and multimodal. These results indicate that sparse multimodal embeddings are superior to their single modality and dense counterparts in their ability to predict interpretable semantic properties corresponding to elements of human conceptual knowledge.