Improving Sparse Word Representations with Distributional Inference for Semantic Composition
Table 6: Neighbour retrieval function. Underlined means best performance per phrase type, boldface means best average performance overall.
['Apt [BOLD] s', '[BOLD] No Distributional Inference  [ITALIC] intersection', '[BOLD] No Distributional Inference  [ITALIC] union', '[BOLD] Density Window  [ITALIC] intersection', '[BOLD] Density Window  [ITALIC] union', '[BOLD] Static Top  [ITALIC] n  [ITALIC] intersection', '[BOLD] Static Top  [ITALIC] n  [ITALIC] union', '[BOLD] WordNet  [ITALIC] intersection', '[BOLD] WordNet  [ITALIC] union']
For a quantitative analysis of distributional inference for semantic composition, we evaluate our model on the composition dataset of Mitchell and Lapata \shortciteMitchell_2010, consisting of 108 adjective-noun, 108 noun-noun, and 108 verb-object pairs. The task is to compare the model’s similarity estimates with the human judgements by computing Spearman’s ρ. For comparing the performance of the different neighbour retrieval functions, we choose the same parameter settings as in the word similarity experiments (k=40 and using 30 neighbours for DI). The density window retrieval function outperforms static top n for composition by intersection and vice versa for composition by union. The WordNet approach is competitive for composition by union, but underperfoms the other approaches for composition by intersection significantly. For further experiments we use the static top n approach as it is computationally cheap and easy to interpret due to the fixed number of neighbours.