Investigation of Sentiment Controllable Chatbot
TABLE VII: Loss of plug and play models with or without KL cost annealing under different epochs on Chinese CECG dataset.
['Epochs KL cost annealing', 'Epochs Without', 'Epochs Total loss', '10000 28.394', '20000 11.163', '30000 5.290', '40000 3.057']
VAE is used for sentence generation in the plug and play model, and KL cost annealing and vocabulary truncation are applied to improve VAE performance. In this experiment, we first compared the performance between models with and without KL cost annealing. These two models truncate vocabulary randomly at a probability of 0.3. This observation shows that the model at first decreases the negative log likelihood instead of improving the KL loss. This is beneficial to the training process once the model gives high priority to the VAE reconstruction.