WordRank: Learning Word Embeddings via Robust Ranking
Table 2: Performance of different ρ functions on Text8 dataset with 17M tokens.
['Task', 'Robi', '[ITALIC] ρ0 off', '[ITALIC] ρ0 on', '[ITALIC] ρ1 off', '[ITALIC] ρ1 on', '[ITALIC] ρ2 off', '[ITALIC] ρ2 on', '[ITALIC] ρ3 off', '[ITALIC] ρ3 on']
In Sec. We now study the utility of these two ideas. For the similarity task, we use the WS-353 test set and for the analogy task we use the Google analogy test set. The best scores for each task are underlined. We set t=1.5 for ρ3. Among the four ρ functions, ρ0 performs the best on the word similarity task but suffers notably on the analogy task, while ρ1=log performs the best overall.