Teacher-Student Framework Enhanced Multi-domain Dialogue Generation
Table 1: The results between the raw model, the model uses GCE as a state tracker and the model uses knowledge distillation. The Seq2seq(Manual states) model takes the manual labeled states as input and sets as the upper bound of our metrics.
['model', 'Multi-domain BLEU', 'Multi-domain Inform(%)', 'Multi-domain Success(%)', 'restaurant BLEU', 'restaurant Inform(%)', 'restaurant Success(%)']
From the table, we can see that in multi-domain our model(HRED-TS) gets the best scores in both the informing rate and the success rate. By adding a teacher-student framework, the informing rate and the success rate receive 4% improvement than the original model. Our model reaches a result as close as the one that takes manual states as input, which is considered as an upper bound. The model with a GCE state tracker increases the Informing rate but has no effect on improving the dialogue success rate. We think that thought the state based on a state tracker has its pros on representing the dialogue processing, the errors exist in the state tracker harms the performance of the model. And our model avoids such a problem by using the teacher-student framework. Results show that our model produces the best results in all models and even outperforms than the one with manual state input. We trust that it is due to using an individual teacher in the restaurant domain while training, which results in a better performance in this domain than the universal one. It is worth noticing that a GCE state has a higher informing rate than the manual state. We believe the lower informing score in manual state one is caused by the influence of state labeling from other domains.