Revisiting Skip-Gram Negative Sampling Model With Rectification
Table 1: Evaluation of SGNS (λ=0) and SGNS-qr models on Google’s analytical reasoning task.
['[ITALIC] d', '[ITALIC] λ 0', '[ITALIC] λ 10', '[ITALIC] λ 50', '[ITALIC] λ 100', '[ITALIC] λ 250', '[ITALIC] λ 500', '[ITALIC] λ 1000']
As we can see clearly, the improvement rate rises from (nearly) 0% to more than 3% quickly as d increases. Remarkably, our simple rectification through quadratic regularization is capable of boosting the prediction accuracy by around 3%.