A Simple Approach to Learn Polysemous Word Embeddings
Table 5: Summary of all quantitative tests and performance metric of our embedding approach against compared baselines. For (Huang et al., 2012) different rows correspond to various types of distance used to get the contextual embedding. For (Neelakantan et al., 2015) the row labels follow the terminology in the paper. We highlight the top 3 results for each test and metric in the table. Sp = Spearman correlation. P@1 = Precision@1. AP = Average Precision. AUC = Area Under Curve. Acc = Classification Accuracy.
['Embeddings', 'dim.', '[BOLD] WCR R1', '[BOLD] WCR R1', '[BOLD] WCR R2', '[BOLD] WCR R2', '[BOLD] WCR R3', '[BOLD] WCR R3', '[BOLD] CWS', '[BOLD] CWS', '[BOLD] SCWS', '[BOLD] WSC C1', '[BOLD] WSC C2']
In this section we validate the performance of our embedding approach on a wide range of tasks that explicitly require contextual understanding of words. the extent of multisense quantitative evaluations (columns 3-6 and 11-13), and supplement them with new, larger test sets (columns 7-9). All tests are provided in the attached dataset folder. In order to keep the evaluation fair across all embeddings, any word that is not in the intersection of the vocabularies of all embeddings is removed from the tests; for the preexisting test sets, this results in slightly smaller test sets than those first proposed. We now describe each task in detail. For each query word-context (w,S) pair, we attach a positive label to another (w′,S′) if S′=S and w, w′ are similar words. We collect negative samples as pairs (w′′,S′′) if S′′≠S, where either w=w′′ or w and w′′ are marked similar words by WordNet in the context S′′. Given a query (w,S) pair, the goal is to rank the similar (positive) pairs above the dissimilar (negative) ones in the context of S. In all, we create a set of 3,955 tests based on 154 polysemous words.