SenGen: Sentence Generating Neural Variational Topic Model
Table 1: Perplexity comparison of various models on two different datasets. All models are configured to use 25 topics. Lower is better.
['Model', '20 Newsgroups', 'CNN/Daily Mail']
We compute perplexity of the test dataset using the trained SenGen model as follows: Perplexity=1NN∑d=1exp(−logP(wd|β)Nd) (9) where the log probability is computed using the lower-bound estimate in Eq. In the above equation, N is the number of test documents, and Nd is the number of words in document d. On 20 Newsgroups datasets, we suspect the main reason is due to the differences in preprocessing between our work and that of others – we noticed that there are many non-dictionary terms in our vocabulary that originated from email signatures and headers. Another potential reason is that the model may be overfitting the training set due to the extremely large number of parameters.