Semantics- and Syntax-related Subvectors in the Skip-gram Embeddings (Student Abstract)
Table 2: Dot products between vectors.
['word  [ITALIC] i', '[BOLD] w⊤dog [BOLD] c [ITALIC] i', '[BOLD] w⊤dog [BOLD] w [ITALIC] i', '[BOLD] x⊤dog [BOLD] x [ITALIC] i', '− [BOLD] y⊤dog [BOLD] y [ITALIC] i']
Consider a phrase the dog barking at strangers Instead, these vectors are split w⊤dog = [x⊤dog;y⊤dog] w⊤barking =[x⊤barking;y⊤barking] in such way that the quantity x⊤dogxbarking−y⊤% dogybarking is large enough. We can interpret this as follows: the word ‘barking’ is semantically close enough to the word ‘dog’ but is not the closest one: e.g. wpuppy is much closer to wdog than wbarking; on the other hand the word ‘barking’ syntactically fits better being next to the word ‘dog’ than ‘puppy’, i.e. −y⊤dogypuppy