Fine-grained Sentiment Controlled Text Generation
Table 1: Controlled generation and Text style transfer accuracy achieved by different methods.
['Methods', 'Yelp Controlled generation', 'Yelp Text style transfer', 'Amazon Controlled generation', 'Amazon Text style transfer', 'IMDB Controlled generation', 'IMDB Text style transfer']
The superior performance of DE-VAE stems from the fact that it learns a disentangled feature representation where sentiment information is modeled in a single dimension which is independent of other dimensions. So, regulating the sentiment value along this designated dimension gives better control while generating sentences bearing the same sentiment. Further, in case of DE-VAE-NR, we train the parameters of the transformation network by maximizing the likelihood of \zbs, i.e. \EEqϕ(\zbs|\xb)pθ(\zbs|\zbf), such that the minimum KL between qϕ(\zbf|\zbs) and pθ(\zbs|\zbf) is found to be in the range 0.2−0.4 across all datasets; however, the performance of DE-VAE-NR is significantly inferior compared to DE-VAE. Close inspection reveals that even though the KL was low, as the decoded distribution \zbs was not exactly the same as the encoded distribution, it was generating sentences very different from the original sentences. This shows the importance of invertible normalizing flow as a design choice. The closest competitor to our method is DAE for controlled text generation with comparable performance to DE-VAE. The reason is that DAE achieves disentanglement between style space and content space using extensive adversarial training and incorporates auxiliary multi-task loss for sentiment or style. Further, in comparison to other methods, ctrlGen performs poorly for controlled text generation. We speculate that ctrlGen has comparatively less control than other methods since it does not model feature representation from sentence representation and tries to control generation by providing only a one hot encoding of sentiment externally. We observe that entangleGen achieves better accuracy on Yelp dataset than our method; however, DE-VAE performs better on all other datasets. Since entangleGen inherently supports binary-valued attributes by design, it can best modify the sentiment value towards its opposite polarity using Fast-Gradient-Iterative-Modification. Hence, its performance suffers in IMDB data where there are more than two sentiment values.