Spherical Paragraph Model
Table 1: Classification accuracies (%) of different models on topical classification.
['Model', 'different', 'similar', 'same', 'Reuters']
We have the following observations: (1) The BoW representation, although simple, can achieve surprising accuracy using much larger dimensionality (i.e., 5,000 dimension). Meanwhile, our SPM, using only 50-dimension text vector, can achieve slightly better or comparable performance as BoW. (2) As compared with the text representation methods built upon BoW (i.e., LSI, LDA, movMF and SAM), SPM can outperform these methods almost. The results indicate that learning text representations over BoWE can in general achieve better performances than that over BoW by involving rich semantics between words. (3) Comparing with the three BoWE based representation methods, namely cBow, PV-DBOW and PV-DM, we find our SPM can outperform them on three out of four datasets. Recall that in cBow, PV-DBOW and PV-DM, texts in a corpus are actually assumed to be independent from each other. These results indicate that by modeling texts under a sound probabilistic generative framework, SPM can well leverage the corpus-wide information to help improve the text representation. (4) Compared with FastSent and uni-skip, SPM can outperform both of them over the four datasets. It seems that FastSent and uni-skip, which were proposed for short texts (i.e., sentences) modeling originally, cannot work well on long texts.