Generating Persona-Consistent Dialogue ResponsesUsing Deep Reinforcement Learning
Table 6: Each row corresponds to the samples in the human evaluation for which SupL-Trans received a particular consistency label. The values in each row show the percentages of consistency labels for DeepRL-Trans for the same data points.
['[EMPTY]', '[EMPTY]', '[ITALIC] DeepRL-Trans label Consistent', '[ITALIC] DeepRL-Trans label Neutral', '[ITALIC] DeepRL-Trans label Contradicting']
For the majority of samples whose SupL-Trans responses are contradictory or neutral, DeepRL-Trans generates consistent responses, confirming the appropriateness of our approach. DeepRL-Trans generates contradictory responses for some samples whose SupL-Trans responses are consistent with their personas. This may be due to errors in the NLI model’s predictions of entailment, hence a more accurate NLI model may improve the quality of the reward function and consequently the consistency of responses. Alternatively, these contradictory responses may receive high rewards from the topic consistency and fluency sub-rewards, which could override R1.