Learning Semantic Relatedness from Human Feedback Using Metric Learning
(b) Delicious
['[EMPTY]', 'MEN', 'WS-353', 'Bib100']
For each line, its header defines the dataset on which the metric was trained, while the column header is the dataset on which the trained metric was then evaluated. In each cell, the first value denotes the Spearman correlation of the cosine measure with the human relatedness scores in the evaluation dataset. The second value is the Spearman correlation of the relatedness scores calculated from the trained metric with the human relatedness scores in the evaluation dataset. Depending on whether the trained metric increased or decreased correlation with human intuition, we depict upwards or downwards arrows. Generally, training a metric on BibSonomy embeddings almost always yields bad transfer results. The only improvement using BibSonomy embeddings is on the WS-353 dataset, when evaluating the metric trained on MEN. The Delicious embeddings are only useful for improving relatedness scores when the trained metric is evaluated on the MEN dataset. The most interesting part of these results is that, using WikiGloVe embeddings, we can always increase correlation with human intuition by a notable margin. However, on the WikiNav embeddings, the results differ only by smaller margins. The only exception here is a notable improvement on the Spearman correlation value of WS-353, when using a metric trained on MEN data. author= chp,color= blue!20todo : author=chp,color =blue!20was sagt uns das jetzt insgesammt? warum ist es nur bei WikiGloVe besser? Overall, it is generally possible to transfer knowledge from one HID to another. However, this is highly dependent on the underlying word representations.