Document Neural Autoregressive Distribution Estimation
Table 4: Test perplexity per word for models with 100 topics. The results for HLBL and LBL were taken from Mnih and Hinton (2009).
['Models', 'Number of grouped sentences', 'Perplexity']
Since language models are generative models, the perplexity measured on some held-out texts provides an intrinsic and widely used evaluation criterion. The dataset is again split into training, validation and test sets, with respectively 633,143, 43,702 and 44,601 sentences. The vocabulary is composed of 17,964 words. A 100 dimensional feature vectors are used for these experiments. These results are meant to measure whether the DocNADE part of DocNADE-LM can indeed help to improve performances.