Generating Sentences from a Continuous Space
Table 2: Penn Treebank language modeling results, reported as negative log likelihoods and as perplexities. Lower is better for both metrics. For the vae, the kl term of the likelihood is shown in parentheses alongside the total likelihood.
['[BOLD] Model', '[BOLD] Standard Train nll', '[BOLD] Standard Train nll', '[BOLD] Standard Train ppl', '[BOLD] Standard Test nll', '[BOLD] Standard Test nll', '[BOLD] Standard Test ppl', '[BOLD] Inputless Decoder Train nll', '[BOLD] Inputless Decoder Train nll', '[BOLD] Inputless Decoder Train ppl', '[BOLD] Inputless Decoder Test nll', '[BOLD] Inputless Decoder Test nll', '[BOLD] Inputless Decoder Test ppl']
The results shown reflect the training and test set performance of each model at the training step at which the model performs best on the development set. Our reported figures for the vae reflect the variational lower bound on the test likelihood, while for the rnnlms, which can be evaluated exactly, we report the true test likelihood. This discrepancy puts the vae at a potential disadvantage.