Improving Word Representations: A Sub-sampled Unigram Distribution for Negative Sampling
Table 4: The results of MSR sentence completion task by previous word representation models and our approach.
['[BOLD] Model', '[BOLD] Acc']
Our focus here is to popularize SWM rather than to compare the noise distributions. The bottom three previous models follow the conventional method. Accordingly, we draw two conclusions: (1) sub-sampling on the words in sentences results in significant improvements to the conventional method; and (2) SWM further improves CMs and beats previous word representation models with a vector size of 300, indicating the success of semantics quantification.