Learning Word Representations with Hierarchical Sparse Coding
Table 1: Summary of results. We report Spearman’s correlation coefficient for the word similarity task and accuracies (%) for other tasks. Higher values are better (higher correlation coefficient or higher accuracy). The last two methods (columns) are new to this paper, and our proposed method is in the last column.
['[ITALIC] M', 'Task', 'PCA', 'RNN', 'NCE', 'CBOW', 'SG', 'SC', 'forest']
In the similarity ranking and sentiment analysis tasks, our method performed the best in both low and high dimensional embeddings. In the sentence completion challenge, our method performed best in the high-dimensional case and second-best in the low-dimensional case. Importantly, forest outperforms PCA and unstructured sparse coding (SC) on every task. We take this collection of results as support for the idea that coarse-to-fine organization of latent dimensions of word representations captures the relationships between words’ meanings better compare to unstructured organization.