Identifying and Explaining Discriminative Attributes
Table 2: Performance comparison (recall) against a random sample of categorized triples. The last row shows the relative gain of the combined model over the best performing individual model.
['[BOLD] Model', '[BOLD] Sensory', '[BOLD] Logical', '[BOLD] Relative', '[BOLD] Absolute', '[BOLD] Essential', '[BOLD] Incidental']
In the experiment, DBM was instantiated using WordNet 3.0 (only glosses structured into its semantic roles), VFM used VisualGenome 1.4 and CKG used ConceptNet 5. In order to perform the evaluation, we selected a stratified random subset of 230 triples and manually classified each triple with the dual categories. A triple can contain one or more categories associated with it. The annotation was performed by two independent annotators, which reached an inter-annotator agreement of 81%. For the final annotated dataset, triples which were not consensual were eliminated (i.e. the final dataset has a 100% inter-annotator agreement). The analysis shows the complementary nature of the models, where DBM contributes more for the identification of the sensory and incidental categories, while CKG contributes to the logical and essential categories. Additionally, VFM provides contributions across all categories and has smaller overlaps with other models.