All-but-the-Top: Simple and Effective Postprocessing for Word Representations
Table 16: Before-After results (x100) on the categorization task.
['[BOLD] Dim', '300 orig.', '300 proc.', '400 orig.', '400 proc.', '500 orig.', '500 proc.', '600 orig.', '600 proc.']
A rule of thumb choice of D is around d/100. We validate this claim empirically by performing the tasks in Section 3 on word representations of higher dimensions, ranging from 300 to 1000, where we set the parameter D=d/100. Again, note that the improvement for analogy tasks is marginal. These experimental results justify the rule-of-thumb setting of D =d/100, although we emphasize that the improvements can be further accentuated by tuning the choice of D based on the specific setting.