Task-Oriented Conversation Generation UsingHeterogeneous Memory Networks
Table 2: Results on Key-Value Retrieval dataset. F1 score, including Entity F1, is micro-average over the entire set, and three subtasks. Human results are reported by Eric et al.DBLP:conf/sigdial/EricKCM17
['Model name', 'BLEU', 'Ent. F1', 'Scheduling Ent. F1', 'Weather Ent. F1', 'Navigation Ent. F1']
The best results of the baselines and HMNs are gathered into tables and figures. Except for F1 scores on Calendar Scheduling, HMNs get significantly better results on all benchmarks comparing the state-of-the-art models. HMMs’ BLEU score is even higher than human results which are reported in DBLP:conf/sigdial/EricKCM17. Results show our model’s outstanding performance in generating a fluent and accurate response in most tasks. , we note that HMNs and Mem2Seq failed on weather forecasting task. We analysed the average knowledge pairs of weather forecasting tasks and find it near three times the knowledge pairs of the other two tasks. Then we carried out another experiment that first narrows the KB candidates by performing a matching preprocessing operation, and the Weather Ent. F1 result of our method will climb to more than 48 which is the best. This may indicates that this kind of memory networks may have difficulties in handling large scale knowledge base. So perform a matching operation to narrow the candidate knowledge space is critical in a real-world large scale knowledge base.