Compositional Approaches for Representing Relations Between Words: A Comparative Study
Table 3: Accuracy of the compositional operators for relational similarity prediction and relational classification (last right column).
['Representation model', 'Compositional operator', 'SAT', 'SemEval', 'MSR', 'Google Sem.', 'Google Syn.', 'Google Total', 'DIFFVECS']
We observe that PairDiff achieves the best results compared with other operators for all the evaluated datasets and all the word representation methods. PairDiff is significantly better than Add or Mult for all embeddings (both prediction- and counting-based) in MSR, Google and DiffVec datasets according to Clopper-Pearson confidence intervals (p < 0.05). SAT is the smallest dataset among all, so we were unable to see any significant differences on SAT. We first train word embeddings of different dimensionalities using the ukWaC corpus. We keep all the other parameters of the word embedding learning method fixed except for the dimensionality of the word embeddings learnt.