Paraphrase-Supervised Models of Compositionality
Table 2: Correlation between model judgments on phrase compositionality and human judgments, measured by Spearman’s ρ. Context-dependent (token-based) and concatenation models do better.
['[BOLD] Setup', '[ITALIC] ρ Add', '[ITALIC] ρ  [BOLD] Concat. ℓ1', '[ITALIC] ρ  [BOLD] Concat. ℓ2']
For the context-independent model, we select the hyperparameter α in Eq. from the values {0.25,0.5,0.75}. For the context-dependent model, we vary the context window size |c| by selecting from the values {4,6,8}. ρ for these setups. In all cases, the context-dependent models outperform the context-independent ones, and using a relatively simple token-based model we can approximately match the performance of the Bayesian model proposed by \newciteHermann2012. The concatenation models are also consistently better than the additive compositional model, indicating the benefit of learning the compositional parameters via PPDB.