1 INTRODUCTION
Table 2: The average number of context words in the co-occurrence matrix according to the type and the size of context.
['[BOLD] TYPE', '[BOLD] DIM.', '[BOLD] SIZE 1', '[BOLD] SIZE 5', '[BOLD] SIZE 10']
First, using all words as context does not imply to reach the best performance. With the 10,000 most frequent words, performance are fairly similar than with all words. An in-between situation with words whose appearance frequency is greater than 10−6 gives also quite similar performance. Secondly, discarding the most frequent words from the context distributions helps, in general, to increase performance. The best performance is indeed obtained with scenarios (3) and (4). But all rare words are not necessarily essential to achieve good performance, since results with words whose appearance frequency is less than 10−5 and greater than 10−6 are not significantly lower. These two observations might be explained by the sparsity of the probability distributions. Similarities between words seem to be easier to find with sparse distributions. The average number of context words (i.e. features) whose appearance frequency is less than 10−5 and greater than 10−6 with a symmetric window of size 1 is extremely low (132). Performance with these parameters are still highly competitive on syntactic tasks. Within this framework, it then becomes a good option for representing words in a low and sparse dimension.