Domain and Function: A Dual-Space Model of Semantic Relations and Compositions
Table 5: Summary of the three spaces.
['Space', 'Rows ( [ITALIC] nr)', 'Columns ( [ITALIC] nc)', 'Nonzeros (after PPMI)', 'Density (after PPMI)']
In the following four sets of experiments, we use the same three matrices (the domain, function, and mono matrices) in all cases; we do not generate different matrices for each set of experiments. Three of the four sets of experiments involve datasets that have been used in past by other researchers. We made no special effort to ensure that the words in these three datasets have corresponding rows in the three matrices. The intention is that these three matrices should be adequate to handle most applications without any special customization. The values in the matrix UkΣpk are typically about half positive and half negative. We use element-wise multiplication as a baseline in some of the following experiments. For a fair baseline, we cannot simply apply element-wise multiplication to row vectors in UkΣpk. One solution would be to use the PPMI matrix, X, which has no negative elements, but this would not allow element-wise multiplication to take advantage of the smoothing effect of SVD. Our solution is to use row vectors from ^X=UkΣkVTk. ^X and UkΣpk have a density of 100%. Another way to deal with element-wise multiplication would be to use nonnegative matrix factorization (NMF) We have not yet found an implementation of NMF that scales to the matrix sizes that we have here In our past experiments with smaller matrices, SVD and NMF have similar performance.