Learning Concept Taxonomies from Multi-modal Data
Table 3: The performance when different combinations of visual features are enabled.
['S-V1', 'PC-V1', 'PC-V2', 'h = 4', 'h = 5', 'h = 6', 'h = 7']
Contributions of visual features. To evaluate the contribution of each part of the visual features to the final performance, we train our model jointly with textual features and different combinations of visual features, and report the ancestor-F1 scores. When incorporating the feature S-V1, the performance is substantially boosted by a large gap at all heights, showing that visual similarity between sibling nodes is a strong evidence for taxonomy induction. It is intuitively plausible, as it is highly likely that two specific categories share a common (and more general) parent category if similar visual contents are observed between them. Further, adding the PC-V1 feature gains us a better improvement than adding PC-V2, but both minor than S-V1.