Context-Dependent Word Representation for Neural Machine Translation
Table 1: The placements of the word embedding vectors along the principals axes in the local charts of “notebook” and “power”. In the case of “notebook”, it is clear that the first axis corresponds to different types of books for writing a note, while the second axis corresponds to portable computing devices. In the case of “power”, the first axis corresponds to political or social authority, while the second axis to physical energy.
['Word \\boldmath [ITALIC] x′', 'Axis', 'Nearest Neighbours']
For any word \boldmathx′ under inspection, we find the N−1 nearest neighbours {\boldmathx1,…,\boldmathxN−1}⊂\boldmathEx in the word embedding matrix. The N word embedding vectors {\boldmathx′,\boldmathx1,…,% \boldmathxN−1} now characterize a local chart centered at \boldmathx′, and we use principal component analysis (PCA) to find the corresponding lower-dimensional Euclidean space. In this Euclidean space, we can inspect the nearest neighbours along each coordinate. et al.