Sketching Word Vectors Through Hashing
Table 2: Differences in Spearman’s ρs obtained by different methods; Hash denotes our method.
['[EMPTY]', 'YP', 'RW', 'M287', 'M771', 'MC30', 'MEN', 'WS353', 'WSS', 'WSR', 'RG65', 'SL']
Concerning our model, disregarding m, our unweighted and PPMI-weighted models outperform their counterpart classic high-dimensional models, as well as the random indexing method. Moreover, for large values of m (e.g., m=1000), our hash-based PPMI-weighted vectors (for most tasks) are a match for the word2vec’s CBOW and GloVe models, particularly when considering the statistical significance of the difference between the observed results from these models, as it is described by \newcite[][see Table 2]rastogi-vandurme-arora:2015: NAACL-HLT and their proposed Minimum Required Difference for Significance (MRDS) measure. Here, we use MRDS to filter out insignificant comparative gains between our method and the baselines. Based on MRDS values in \newciterastogi-vandurme-arora:2015: NAACL-HLT, while our method lags behind CBOW and GloVe in \NAT@swafalse\NAT@partrue\NAT@fullfalse\NAT@citetpHill2015 SL, it certainly performs better than its neural counterparts in the RW and Wsr tests. Similarly, MRDS suggests that in the Men test, the CBOW clearly outperforms all the other methods, whereas the difference in the obtained ρs by our method and GloVe is insignificant. By the same token, GloVe and CBOW both perform equally well over most of the tests except MEN and RW, for which CBOW and GloVe perform better than the other, respectively.