“The Sum of Its Parts”:Joint Learning of Word and Phrase Representations with Autoencoders
Table 2: Phrase representations evaluation. Comparison of performance across all models with 100-dimensional phrase vector representations on word retrieval. R@K is Recall@K, with K={1,5,10}.
['[EMPTY]', '[BOLD] R@1', '[BOLD] R@5', '[BOLD] R@10']
Our model significantly outperforms others methods on this task. However GloVe’s representations cannot maintain this performance for longer phrases. It is probably not too surprising as this model is trained using ratios of co-occurrence probabilities for two target words. Consequently, it well learns linear substructures for pairs of words, which probably also explains its good performance on word analogy tasks. In contrast, our joint model can learn more complex substructures which make possible the aggregation of multiple words within a low-dimensional vector space.