Entity-Consistent End-to-end Task-Oriented Dialogue System with KB Retriever
Table 1: Comparison of our model with baselines
['Model', 'InCar BLEU', 'InCar F1', 'InCar Navigate F1', 'InCar Weather F1', 'InCar Calendar F1', 'CamRest BLEU', 'CamRest F1']
Follow the prior works Eric et al. ; Wen et al. We argue that their results are not directly comparable because their work uses the entities in thier canonicalized forms, which are not calculated based on real entity value. It’s noticing that our framework with two methods still outperform KV Net in InCar dataset on whole BLEU and Entity F metrics, which demonstrates the effectiveness of our framework. Our model outperforms each baseline on both BLEU and F1 metrics. In InCar dataset, Our model with Gumbel-Softmax has the highest BLEU compared with baselines, which which shows that our framework can generate more fluent response. Especially, our framework has achieved 2.5% improvement on navigate domain, 1.8% improvement on weather domain and 3.5% improvement on calendar domain on F1 metric. It indicates that the effectiveness of our KB-retriever module and our framework can retrieve more correct entity from KB. In CamRest dataset, the same trend of improvement has been witnessed, which further show the effectiveness of our framework.