HyperLex: A Large-Scale Evaluation of Graded Lexical Entailment
Table 3: Example word pairs from HyperLex. The order of words in each pair is fixed, e.g., the pair chemistry / science should be read as “Is CHEMISTRY a type of SCIENCE?”
['Pair', 'HyperLex LE Rating']
S4SS0SSS0Px2 Final Output The HyperLex evaluation set contains noun pairs (2,163 pairs) and verb pairs (453 pairs) annotated for the strength of the lexical entailment relation between the words in each pair. Since the LE relation is asymmetric and the score always quantifies to what degree X is a type of Y, pairs (X,Y) and (Y,X) are considered distinct pairs. Each concept pair is rated by at least 10 human raters. The rating scale goes from 0 (no type-of relationship at all) to 10 (perfect type-of relationship). Several examples from HyperLex are provided in Tab.