Learning Semantic Relatedness from Human Feedback Using Metric Learning
Table 2: Spearman correlation scores for both the high-dimensional representation (ρhigh) and word embeddings (ρemb) of both tagging datasets. It can be seen that the word embeddings encode semantic relations which are more in line with human judgment than the high-dimensional representations.
['datasets', 'BibSonomy  [ITALIC] ρhigh', 'BibSonomy  [ITALIC] ρemb', 'Delicious  [ITALIC] ρhigh', 'Delicious  [ITALIC] ρemb']
We evaluate our approach on several domains. This includes semantic relatedness extracted from tagging data. However, vector representations of words extracted from tagging data are traditionally high-dimensional [cattuto2008semantic, markines2009evaluating], making metric learning in this domain infeasible due to a more than quadratic runtime with regard to the number of vector dimensions [liu2012metric]. Thus, similar to our Wikipedia examples we employ the notion of (low-dimensional) word embeddings which have been shown to outperform their high-dimensional counterparts in terms of correlation with human intuition of semantic relatedness [baroni2014count]. This can also be confirmed when using tagging data as input In this section, we justify our choice of using GloVe to embed words based on tagging data and study the influence of dimensionality on the respective semantic content.