Definition Frames: Using Definitions for Hybrid Concept Representations
Table 5: Spearman’s correlation for embeddings before and after the linear transform. We show in bold the model with the best performance and the highest Gain. We denote with ∗ the experiments with p-value p<0.001.
['[BOLD] Dataset', '[BOLD] GloVe Basis', '[BOLD] GloVe Basis∗', '[BOLD] GloVe  [ITALIC] Gain', '[BOLD] GloVe DF', '[BOLD] GloVe DF∗', '[BOLD] GloVe  [ITALIC] Gain', '[BOLD] Dict2Vec Basis', '[BOLD] Dict2Vec Basis∗', '[BOLD] Dict2Vec  [ITALIC] Gain', '[BOLD] Dict2Vec DF', '[BOLD] Dict2Vec DF∗', '[BOLD] Dict2Vec  [ITALIC] Gain']
For our experiments we use 10-Fold cross-validation and we report the average performance. We ignore datasets with less than 100 instances due to their small size. We also follow the same method for the Basis embeddings on each dataset by learning the parameters Wbasis, bbasis. Since they were the best performing embeddings in the previous section, we perform experiments with both GloVe and dict2vec as the Basis embeddings used for Basis and DF. The performance of the embeddings before and after the transformation is reported on the same cross-validation splits to avoid randomness. Finally, for our reported results, we ignore datasets where both DF and Basis embeddings show lower performance after the linear transformation (MTurk287, MTurk771 and RW-STAN) or with a high p-value (p>0.05) for the cross validation splits (SimVerb), as this hints inconsistency of the type of similarity within the dataset.