Neural-based Noise Filtering from Word Embeddings
Table 1: Effects of word denoising embeddings. Vectors X represent the baselines; vectors A and B were suggested by Faruqui:15; the vector length Z∗ is equal to 10 times of vector length X.
['[BOLD] Vectors', '[BOLD] Vectors  [BOLD] X', '[BOLD] Simlex-999 Corr. 33.7', '[BOLD] MEN Corr. 72.9', '[BOLD] WS353 Corr. 69.7', '[BOLD] WS353-SIM Corr. 74.5', '[BOLD] WS353-REL Corr. 65.5', '[BOLD] ESL Acc. 48.9', '[BOLD] TOEFL Acc. 62.0', '[BOLD] NP Acc. 72.8']
The results show that the vectors X∗ and Z∗ outperform the original vectors X,A and B, except for the NP task, in which the vectors B based on the 300-dimensional GloVe vectors are best. The effect of the vectors Z∗ is slightly less impressive, when compared to the overcomplete vectors X∗. The overcomplete word embeddings Z strongly differ from the word embeddings X; hence, the denoising is affected. However, the performance of the vectors Z∗ still outperforms the original vectors X,A and B after the denoising process.