A Dual Reinforcement Learning Framework for Unsupervised Text Style Transfer
Table 4: Human evaluation results on full model (RL+MLE) and ablated models on the Yelp dataset.
['[EMPTY]', 'Sty', 'Con', 'Flu', '[BOLD] Avg', '[BOLD] Suc']
In this section, we give a deep analysis of the key components of our model. It shows if we only train the model based on RL, the ACC (Sty) will increase, while the BLEU (Con, Flu) will decline. The reason is that the RL training may encourage the model to generate tricky sentences which can get the high reward but fails in quality and readability (measured by BLEU). For example, given a negative review “We sit down and got some really slow and lazy service” as the input, the model may generate “We sit down and got some really great and great service”. This output sentence is not fluent but it can get a high style classifier reward and content preservation reward, thus leading the model to train towards a bad direction. In contrast, the Seq2Seq model using MLE objective is essentially a conditional language model which can ensure readability of the output, thus showing higher BLEU (Con, Flu) score than RL. However, the ACC (Sty) of MLE declines, since there is no specific mechanism in MLE to directly control the style accuracy. Finally, the combination of RL and MLE can get best BLEU score without compromising ACC, with over 3.5 points absolute improvement in H2/G2 score and 13% more success rate (Suc) than model trained only based on MLE.