Collaborative Training of Tensorsfor Compositional Distributional Semantics
Table 1: Top: Spearman values as the percentage of adjectives with training data is increased (EX1). Bottom: Spearman values as the percentage of training data per adjective is increased (EX2). Grey cells indicate values were used as test data during parameter tuning, while grey numbers indicate non-significant correlations. Highest values in each section and column are bolded.
['[BOLD] Adjective Experiment 1 (EX1)  [BOLD] Full Tensor', '[BOLD] Adjective Experiment 1 (EX1)  [BOLD] ML10', '[BOLD] Adjective Experiment 1 (EX1)  [BOLD] ML10', '[BOLD] Adjective Experiment 1 (EX1)  [BOLD] ML10', '[BOLD] Adjective Experiment 1 (EX1)  [BOLD] ML10', '[BOLD] Adjective Experiment 1 (EX1)  [BOLD] ML10', '[BOLD] Adjective Experiment 1 (EX1)  [BOLD] MEN', '[BOLD] Adjective Experiment 1 (EX1)  [BOLD] MEN', '[BOLD] Adjective Experiment 1 (EX1)  [BOLD] MEN', '[BOLD] Adjective Experiment 1 (EX1)  [BOLD] MEN', '[BOLD] Adjective Experiment 1 (EX1)  [BOLD] MEN', '[BOLD] Adjective Experiment 1 (EX1)  [BOLD] SIMLEX', '[BOLD] Adjective Experiment 1 (EX1)  [BOLD] SIMLEX', '[BOLD] Adjective Experiment 1 (EX1)  [BOLD] SIMLEX', '[BOLD] Adjective Experiment 1 (EX1)  [BOLD] SIMLEX', '[BOLD] Adjective Experiment 1 (EX1)  [BOLD] SIMLEX']
In the grey cells we have the values which were considered when choosing the parameter settings. The rest of the values in the tables represent testing results without tuning on these datasets. The light grey numbers are not significantly correlated with the data with p<0.05. Some negative correlations occur in the testing results (see Sec. GLF performs well when there is moderate sparsity in the percentage of available adjectives (5%-70%). PS+FT gets highest results on MEN and SIMLEX more often than GLF, and most significantly both GLF and PS+FT regularly outperform LF even when all the training data is available. Since the goal of training tensors for the Categorial Framework is good performance in composition tasks, we tuned our parameters on an adjective-noun composition dataset (ML10). we have acceptable performance on this dataset; but, the performance on the datasets where we compare the adjective matrices directly (MEN and SIMLEX) is statistically non-correlated and sometimes negative. Nearest neighbours analysis of adjectives produced by the parameter settings fix1 and var shows that similarities between all adjectives approach one and lead to nonsensical rankings. This is due to the fact that the numbers in the matrices themselves are very low and approach machine precision. Using cosine similarity leads to elementwise multiplication between low numbers and hence near-zero values in the numerator. On the other hand, composition is performed using matrix multiplication between the adjective matrices and the noun vectors, which have much larger numbers, resulting in the more sensible performance in the composed datasets.