Dynamic Knowledge Routing Network For Target-Guided Open-Domain Conversation
Table 4: Results of Self-Play Evaluation.
['System', 'TGPC Succ. (%)', 'TGPC #Turns', 'CWC Succ. (%)', 'CWC #Turns']
Figure 2: A conversation example between human (H) and agents (A) with the same target and starting utterance in CWC dataset.Keywords predicted by the agents or mentioned by human are highlighted in bold. The target achieved at the end of a conversation is underlined. judge whether the current conversation context contains the end target. We set a maximum number of turns as 8 to prevent from an endless conversation that can not reach the target. We use the success rate of reaching the targets (Succ.) and the average number of turns taken to achieve the target (#Turns) as our evaluation criterion. Table 4 shows the results of 500 simulations for each of the comparison systems. Although the average number of turns of our approach is slightly more than Kernel, our system obtains the highest success rate, significantly improving over other approaches. We conduct user study for a more thorough on our CWC dataset. We use the DialCrowd e et al. 2018) to set up user study interfaces and kinds of user studies as below. The first study sure system performance in terms of two key amely transition smoothness and target achieveonstruct 50 test cases, each of which has a target g utterance. In each case, a user will chat with a lected agent. If the agent thinks the conversation the given target or the maximum number of conrns, the agent will inform and show the target to e DialCrowd will ask the user to judge whether reached and rate the transition smoothness of tion with a smoothness score. The smoothness s from 1 (very bad) to 5 (very good). All agents ur DKRN agent outpe Table 6 shows the results of the second study outperforms the comparison agents with a large example, compared to Kernel, 62% users deter agent is better on CWC dataset. This superior confirms the effectiveness of our approach. Qualitative Study To more intuitively demonstrate the superior per our agent, two conversations in user study deriv kernel agent and our DKRN agent respe Figure 2. Both conversations have the same targ starting utterance. From Figure 2, we can see th than Kernel, a achieve the target. The complete conversations o line agents and our DKRN agent are presented plemental material. Conclusions In this work, we move forward to develop an effe guided open-domain conversation system that a chat with user target. Spec routing net semantic knowledge relations among candidat into turn-level keyword prediction for the smoot sition. We also propose a simple but effective dua