Sparse Overcomplete Word Vector Representations
Таблица 3: Performance comparison of transformed vectors to initial vectors X. We show sparse overcomplete representations A and also binarized representations B. Initial vectors are discussed in §A and tasks in §B.
['Vectors', 'Vectors', 'SimLex Corr.', 'Senti. Acc.', 'TREC Acc.', 'Sports Acc.', 'Comp. Acc.', 'Relig. Acc.', 'NP Acc.', 'Average']
First, we quantify the effects of our transformations by comparing their output to the initial (X) vectors. The exceptions are on the SimLex task, where our sparse vectors are worse than the skip-gram initializer and on par with the multilingual initializer. Sparsification is beneficial across all of the text classification tasks, for all initial vector representations. On average across all vector types and all tasks, sparse overcomplete vectors outperform their corresponding initializers by 4.2 points. On average across all vector types and all tasks, binarized overcomplete vectors outperform their corresponding initializers by 4.8 points and the continuous, sparse intermediate vectors by 0.6 points.