HyperLex: A Large-Scale Evaluation of Graded Lexical Entailment
Table 15: Results on the two Test sets of HyperLex splits with a selection of unsupervised LE models. Lower scoring model variants are not shown.
['Model', '[BOLD] Random', '[BOLD] Lexical']
, we rely on the selection of state-of-the-art word embedding models to provide feature vectors →X and →Y. The results of a variety of tested regression models are summarised in Fig. As another reference point, we also report results with several unsupervised models on the two smaller test sets in Tab. The IAA scores from Tab. The scores on the smaller test sets follow similar patterns as on the entire HyperLex. We see a slight increase of performance for similarity-specialised models (e.g., WN-based models or Paragram+CF) on the lexical split. We attribute this increase to the larger percentage of verb pairs in the lexical test set, shown to be better modelled with similarity-oriented embeddings in the graded LE task. Verb pairs constitute 17.3% of the entire random test set, the same percentage as in the entire HyperLex, while the number is 26.4% for the lexical test set.