[
Table 2: Results for the Vector Addition Baseline, Smoothed Vector Addition Baseline, LSTM and the Structured Distributional Model (SDM) on the RELPRON development set (Mean Average Precision scores). Rows refer to the different word combinations tested in Rimell et al. 2016 (R&al.).
['[EMPTY]', '[BOLD] Word combination', '[BOLD] R&al.', '[BOLD] SG', '[BOLD] CBOW', '[BOLD] C-Phrase']
First of all, the Skip-Gram based models always turn out to be the best performing ones, with rare exceptions, closely followed by the C-Phrase ones. The scores of the additive models are slightly inferior, but very close to those reported by Rimell et al. Results seem to confirm the original findings: even with very complex models (in that case, the Lexical Function Model by Paperno et al. 2014), it is difficult to outperform simple vector addition in compositionality tasks. All the results for the headNoun+verb+arg composition are, to the best of our knowledge, the best scores reported so far on the dataset. Unfortunately, given the relatively small size of RELPRON, the improvement of the gek models fails to reach significance (p>0.1 for all comparisons between a basic additive model and its respective augmentation with deg, p−values computed with the Wilcoxon rank sum test). We take these results as supporting the added value of the structured event knowledge and the sr update process in SDM, over the simple enrichment of vector addition with nearest neighbors. Finally, we can notice that the Skip-Gram vectors have again an edge over the competitors, even over the syntactically-informed C-Phrase vectors.