Syntactic Interchangeability in Word Embedding Models
Table 2: Percentage of interchangeable neighbors per pivot POS for the smallest (1) and largest (15) windows in our experiment, for the CBOW and SGNS algorithms. The number of interchangeable neighbors has a strong negative Pearson correlation (r) with window size for windows 1 to 15 (p<0.01, two-tailed t-test).
['[BOLD] Algo-  [BOLD] rithm', '[BOLD] NOUN 1', '[BOLD] NOUN 15', '[BOLD] NOUN r', '[BOLD] ADJ 1', '[BOLD] ADJ 15', '[BOLD] ADJ r', '[BOLD] VERB 1', '[BOLD] VERB 15', '[BOLD] VERB r']
For nouns, adjectives and verbs, we consistently see a decrease in the number of same-POS neighbors when we increase the window size, relative to the total number of nearest neighbors.