SimVerb-3500: A Large-Scale Evaluation Set of Verb Similarity
Table 4: Spearman’s ρ correlation between human judgments and model’s cosine similarity by VerbNet Class. We chose classes #13 Verbs of Change of Possession, #31 Verbs of Psychological State, #37 Verbs of Communication, #45 Verbs of Change of State, and #51 Verbs of Motion as examples. All are large classes with more than 100 pairs each, and the frequencies of member verbs are distributed in a similar way.
['[BOLD] Model', '#13', '#31', '#37', '#45', '#51']
S6SS0SSS0Px3 VerbNet Classes Another analysis enabled by SimVerb-3500 is investigating the connection between VerbNet classes and human similarity judgments. We find that verbs in the same top-level VerbNet class are often not assigned high similarity score. Out of 1378 pairs where verbs share the top-level VerbNet class, 603 have a score lower than 5. Tab. When a verb belongs to multiple classes, we count it for each class (see Footnote 2). We run the analysis on the five largest VN classes, each with more than 100 pairs with paired verbs belonging to the same class.