Sketching Word Vectors Through Hashing
Table 1: Performance of our method in semantic similarity and relatedness tasks and its comparison to baselines. The last two columns report arithmetic and geometric mean of the observed performances (i.e., the harmonic mean of ρ and r) obtained across tasks. For the hash-based method, we report results for both unweighted and PPMI-weighted vectors of various dimensionality. We use Goodman and Kruskal’s γ and Pearson’s r for estimating similarities for unweighted and PPMI weighted vectors build by our method, respectively. For baselines, we use cosine as the similarity estimator.
['[EMPTY]', '[BOLD] Vector Set', '[BOLD] Vector Set', '[BOLD] Similarity Tests  [BOLD] YP130', '[BOLD] Similarity Tests  [BOLD] RW', '[BOLD] Similarity Tests  [BOLD] M287', '[BOLD] Similarity Tests  [BOLD] M771', '[BOLD] Similarity Tests  [BOLD] MC30', '[BOLD] Similarity Tests  [BOLD] MEN', '[BOLD] Similarity Tests  [BOLD] WS353', '[BOLD] Similarity Tests  [BOLD] WSS', '[BOLD] Similarity Tests  [BOLD] WSR', '[BOLD] Similarity Tests  [BOLD] RG65', '[BOLD] Similarity Tests  [BOLD] SL', 'Mean  [BOLD] A', 'Mean  [BOLD] G']
For our method, we build word vectors of dimension m= 500, 1000, 2000, and 4000, and report results obtained from both the unweighted vectors—with γ (Eq. (Eq. Obviously, increasing the dimension of our vector does not affect the computational complexity (nor the required training time) for building a model; however, it increases the cost of similarity computation. Figure LABEL: dimensionAvgPerformance plots changes in the method’s performance (i.e., the arithmetic average of performances across tasks in our experiments) when the dimensionality of our model is increased— the evaluation setup (i.e., the input corpus and context window size) remains the same as those reported earlier. As shown, up to m=10,000, the performance mostly improves. We admit that increasing dimensionality is not desirable due to the cost that it imposes in the subsequent process (e.g., estimating pairwise similarities). However, we notice that these models of higher dimensionality (e.g., m=2000) can be compressed (e.g., to m=250) using matrix factorization techniques (e.g., PCA) without hurting task-based performances. Since our models are already of low-rank, factorization (e.g., computing principal components) can be done considerably fast.