Sentence Level Recurrent Topic Model: Letting Topics Speak for Themselves
Table 1: Average test perplexity per word of different models. Lower perplexity means better document modelling.
['[ITALIC] Dataset\\  [ITALIC] Model', '20Newsgroup  [ITALIC] K=128', '20Newsgroup  [ITALIC] K=256', 'Wiki10+  [ITALIC] K=128', 'Wiki10+  [ITALIC] K=256']
We measure the performances of different topic models according to the perplexity per word on the test set, defined as perp(D)=exp{−∑Mi=1logP(di)∑Mi=1N′i}, where N′i is the number of words in document di. The experimental results are summarized in Based on the table, we have the following discussions: Our proposed SLRTM consistently outperforms the baseline models by significant margins, showing its outstanding ability in modelling the generative process of documents. Models that consider sequential property within sentences (i.e., GMNTM and SLRTM) are generally better than other models, which verifies the importance of words’ sequential information. Furthermore, LSTM-RNN is much better in modelling such a sequential dependency than standard feed-forward networks with fixed words window as input, as verified by the lower perplexity of SLRTM compared with GMNTM.