EmpGAN: Multi-resolution Interactive Empathetic Dialogue Generation
Table 1: Automatic evaluation results between models. Bold face indicates leading results in terms of the corresponding metric. The first five lines are the baselines with EmpGAN. EmpG: Empathetic generator; EmpD: EmpG with Vanilla GAN; EmpWD-next: EmpG with semantic discriminator without user feedback; EmpWD: EmpG without emotional discriminator; EmpGAN: the whole model.
['[BOLD] Models', '[BOLD] BLEU', '[BOLD] Distinct  [BOLD] D-1', '[BOLD] Distinct  [BOLD] D-2', '[BOLD] ROUGE-1  [BOLD] F', '[BOLD] ROUGE-1  [BOLD] P', '[BOLD] ROUGE-1  [BOLD] R', '[BOLD] ROUGE-2  [BOLD] F', '[BOLD] ROUGE-2  [BOLD] P', '[BOLD] ROUGE-2  [BOLD] R', '[BOLD] ROUGE-L  [BOLD] F', '[BOLD] ROUGE-L  [BOLD] P', '[BOLD] ROUGE-L  [BOLD] R']
For research question RQ1, we examine the overall performance in term of BLEU, Distinct, ROUGE and human evaluation. That is, the responses it generates perceive as more natural and diverse. One note is that the three metrics of the models considering emotional factors are generally higher than the general models, which means that incorporating emotional factors into models is a more natural way to learn the goal distribution. We observe that EmpGAN achieves a 7%, 129% and 29% increment over the state-of-the-art baseline Emo-HRED in terms of BLEU, Distinct and ROUGE score, respectively. Specifically, EmpG is on par with Emo-HRED on BLEU, while EmpG achieves a noticeable improvement on Distinct. This observation demonstrates that, compared with Emo-HRED that considers only emotional labels, empathetic generator efficiently takes advantage of multi-granularity emotional factors and sufficiently perceives emotional context resulting in more diversified responses. The comparison between EmpG and Emo-HRED on the three metrics has proved the helpfulness of multi-resolution mechanism for the model performance. When compared with EmpG, EmpGAN achieves 6.4, 3.69 and 1.82 gains on BLEU, Distinct and ROUGE respectively, which verifies the effectiveness of interactive discriminators that interact with responses, overall context and user feedback. In the model EmpD, we adopt the Vanilla GAN where the outputs from a discriminator are used as rewards for the EmpG. We obtain a small increment from EmpG to EmpD in terms of BLEU and Distinct, which demonstrates the effectiveness of discriminator. Another improvement from EmpD to EmpWD-next indicates the performance of EmpWD-next benefits from Wasserstein adversarial learning with gradient penalty. When injecting the user feedback into the discriminator, the EmpWD achieves a 0.18, 0.85 and 0.25 improvement over EmpWD-next on BLEU, Distinct and ROUGE. We conclude that user feedback as an additional signal helps the discriminator generate more efficient outputs by interacting with responses and context to facilitate empathetic generator optimization. Comparing EmpGAN and EmpWD, EmpWD shows a comparable performance on Distinct, whereas the performance on BLEU and ROUGE decrease, which partially verifies that the emotional discriminator induces the model generate more grammatical responses than before.