Embedding Semantic Relations into Word Representations
Table 1: Word analogy results on benchmark datasets.
['Method', 'sem.', 'synt.', 'total', 'SAT', 'SemEval']
In all of our experiments, the proposed method converged with less than 5 iterations. analogy questions in the Google dataset using CosMult measure. For analogy questions in SAT and SemEval datasets the best performance is reported by the proposed method using the PairDiff measure. The PairDiff measure computes the cosine similarity between the two difference vectors →b−→a and →d−→c, ignoring the spatial distances between the individual words as opposed to CosAdd or CosMult. Recall that in the Google dataset we are required to find analogies from a large open vocabulary whereas in SAT and SemEval datasets the set of candidates is limited to a closed pre-defined set. Relying on direction alone, while ignoring spatial distance is problematic when considering the entire vocabulary as candidates because, we are likely to find candidates d that have the same relation to c as reflected by →a−→b. For example, given the analogy man:woman::king:?, we are likely to recover feminine entities, but not necessarily royal ones using PairDiff. On the other hand, in both SemEval and SAT datasets, the set of candidate answers already contains the related candidates, leaving mainly the direction to be decided. For the remainder of the experiments described in the paper, we use CosMult for evaluations on the Google dataset, whereas PairDiff is used for the SAT and SemEval datasets. that according to the binomial exact test with 95% confidence the proposed method statistically significantly outperforms GloVe, the current state-of-the-art word representation learning method, on all three benchmark datasets.