The Effects of Data Size and Frequency Rangeon Distributional Semantic Models
Table 1: Results for DSMs trained on data of varying sizes.
['[BOLD] DSM', '[BOLD] TOEFL', '[BOLD] ESL', '[BOLD] SL', '[BOLD] MEN', '[BOLD] RW']
The most notable aspect of these results is that the neural networks models do not produce competitive results for the smaller data, which corroborates the results by Asr et al. \shortciteAsr:2016. The best results for the smallest data are produced by the factorized models, with both tsvd and isvd producing top scores in different test settings. It should be noted, however, that even the top scores for the smallest data set are substandard; only two models (ppmi and tsvd) manage to beat the random baseline of 25% for the TOEFL tests, and none of the models manage to beat the random baseline for the ESL test.