Understanding Composition of Word Embeddings via Tensor Decomposition
Table 9: Test accuracy for sentiment analysis task (standard deviation reported in parentheses)
['Dataset', 'Additive', 'Tensor']
We test the effect of using the composition tensor for a sentiment analysis task. Maas et al. For a fixed review, we identify each adjective-noun pair (a,b) and compute T(va,vb,⋅). We add these compositions together with the word embeddings for all of the words in the review, and then normalize the resulting sum. This vector is used as the input to a regularized logistic regression classifier, which we train using scikit-learn (Pedregosa et al. with the default parameters. We also consider a baseline method where we simply add together all of the word embeddings in the movie review, and then normalize the sum. We evaluate the test accuracy of each method using 5-fold cross-validation on the smaller dataset and the training-test set split provided in the larger dataset. Although the tensor method seems to have a slight edge over the baseline, the differences are not significant.