Hierarchical Context Enhanced Multi-Domain Dialogue System for Multi-domain Task Completion
Table 5: Final results of the DSTC-8 HUMAN evaluation
['Rank', 'Team Submission ID', 'Success Rate', 'Language Understanding Score', 'Response Appropriateness Score', 'Turns']
Our HCEDS also achieves significantly better performance than the baseline in the evaluated four metrics. More specifically, the HCEDS achieves a 65.81% task success rate and a 16.6% improvement over the baseline system. The system’s NLU score reached 3.538, an increase of 14.2% compared with the baseline. The response appropriateness score is 3.632, 2.1% higher than the baseline. The average number of dialogues was around 15, which was reduced by about 2 turns compared with 17 turns of the baseline. Overall, we have a significant improvement over the baseline in both NLU and the average rounds.