Can Network Embedding of Distributional Thesaurus be Combined with Word Vectors for Better Representation?
Table 7: Comparison of accuracies between GloVe representation, DT embedding using node2vec and combination of both where PCA is the combination technique. Clearly DT embedding is helping to improve the performance of GloVe for synonym detection as well as analogy detection.
['[BOLD] Dataset', '[BOLD] GloVe', '[BOLD] D2V-N', '[BOLD] PCA (GloVe, D2V-N)']
We consider two gold standard datasets for the experiment of synonym detection. The descriptions of the used datasets are given below. TOEFL: It contains 80 multiple-choice synonym questions (4 choices per question) introduced by Landauer and Dumais Being consistent with the previous experiments, we consider only nouns for our experiment and prepare TOEFL-N which contains 23 synonym questions. ESL: Here also we consider only nouns for our experiment and prepare ESL-N which contains 22 synonym questions. Note that, in our experimental setup we do not use the context per question provided in the dataset for evaluation. While preparing both the datasets we also keep in mind the availability of word vectors in both downloaded GloVe representation and prepared DT embedding. For evaluation of the word embeddings using TOEFL-N and ESL-N, we consider the option as the correct answer which is having highest cosine similarity with the question and report accuracy. s=a1.a2+b1.b2+w1(b2−a2).(b1−a1)+w2(b2−b1).(a2−a1) As mentioned by the authors, the appropriate values of w1 and w2 is optimized separately for each system using grid search, to achieve the best performance. We use accuracy as the evaluation metric. Note that, the optimized values of (w1,w2) are (0.2,0.2), (0.8,0.6), (6,0.6) for GloVe, DT embedding, combined representation of GloVe and DT embeddings respectively, for the analogy task.