Sparse Overcomplete Word Vector Representations
Таблица 1: Hyperparameters for learning sparse overcomplete vectors tuned on the WS-353 task. Tasks are explained in §B. The four initial vector representations X are explained in §A.
['[BOLD] X', '[ITALIC] L', '[ITALIC] λ', '[ITALIC] τ', '[ITALIC] K', '% Sparse']
Methods A and B have three hyperparameters: the ℓ1-regularization penalty λ, the ℓ2-regularization penalty τ, and the length of the overcomplete word vector representation so that the vectors in D are near unit norm. These hyperparameters were chosen for method A and retained for method B. Using methods A and B, we constructed sparse overcomplete vector representations A and B resp., The first of these is a word similarity task, where the score is correlation with human judgments, and the others are classification accuracies of an ℓ2-regularized logistic regression model trained using the word vectors.