Generating Sentences from a Continuous Space
Table 4: Results for adversarial evaluation of imputations. Unigram and lstm numbers are the adversarial error (see text) and rnnlm numbers are the negative log-likelihood given to entire generated sentence by the rnnlm, a measure of sentence typicality. Lower is better on both metrics. The vae is able to generate imputations that are significantly more difficult to distinguish from the true sentences.
['[BOLD] Model', '[BOLD] Adv. Err. (%) Unigram', '[BOLD] Adv. Err. (%) lstm', '[BOLD] NLL rnnlm']
S5SS0SSS0Px3 Results As a consequence of this experimental setup , the rnnlm cannot choose anything outside of the top 15 tokens given by the rnn’s initial unconditional distribution P(x1|Null) when producing the final token of the sentence, since it has not yet generated anything to condition on, and has a beam size of 15. Additionally, we include the score given to the entire sentence with the imputed completion given a separate independently trained language model. The likelihood results are comparable, though the rnnlms favoring of generic high-probability endings such as “he said,” gives it a slightly lower negative log-likelihood. Measuring the rnnlm likelihood of sentences themselves produced by an rnnlm is not a good measure of the power of the model, but demonstrates that the rnnlm can produce what it sees as high-quality imputations by favoring typical local statistics, even though their repetitive nature produces easy failure modes for the adversarial classifier. Accordingly, under the adversarial evaluation our model substantially outperforms the baseline since it is able to efficiently propagate information bidirectionally through the latent variable.