Stylized Text Generation Using Wasserstein Autoencoders with a Mixture of Gaussian Prior
Table 1: Comparison between our work and others. Note that the classification accuracy for separate WAE models is not a valid measure since each model is only trained on a single class. Hence the accuracy should be 100% in theory. Models with (s) in their names, are trained on a small subset of MNLI with 40960 raining samples for four classes
['[EMPTY]', 'D-1↑', 'D-2↑', 'Entropy↑', '(PPL)↓', 'Classification↑']
Style Accuracy : We follow the approach of the previous work hu2017toward, shen2017style, fu2018style, john2018disentangled and separately train a convolutional neural network (CNN) to classify sentences kim2014convolutional based on their classes. We use this classifier to classify sentences generated with our approach and compare our results with a separate WAE trained only on a specified class of a dataset. We also ran our experiments with separate VAEs and the results are very close to separate WAEs. The classification accuracy of the classifier over the original MNLI dataset is 98%. Using the WAE with a GM prior lowers the accuracy of the classifier. This is expected because we use a shared decoder over multiple class distributions, but the classifier still is easily able to identify different classes. Perplexity: We use the Kneser-Ney language model kneser1995improved to evaluate the fluency of our sampled sentences. We measure the empirical distribution of trigrams in a corpus, and compute the log-likelihood of a sentence. We train the language model on the original dataset and evaluate the fluency of our sampled sentences. Our model achieves acceptable results compared to separate WAEs trained on only one class of sentences. This is again because we are sharing the decoder for all of the GM priors and thus, the decoder sometimes uses rare words to generate a sentence for a specified class. These rare words might be common words in another class. Since the language model has not seen such a combination of words, it evaluates these sentences with a lower accuracy. Note that smaller values correspond to more fluent sentences. Diversity: We use distinct diversity metrics by computing the percentage of distinct unigrams and bigrams following the work of li2015diversity and bahuleyan2017variational. Our model outperforms all of the baselines and previous models in terms of sentence diversity. This is because the decoder learns on a more diverse set of sentences when it is trained over multiple classes. For the question generation task, since the dataset is very small, neither of the models are successful at generating diverse sentences and they tend to generate the same set of question over and over. However, WAE with GM prior generates twice more diverse sentences compared to separate WAEs.