Teacher-Student Framework Enhanced Multi-domain Dialogue Generation
Table 3: The results between adding different components in a distillation step. The last column shows the results of distilling from a universal teacher model.
['distill part', 'Multi-domain BLEU', 'Multi-domain Inform(%)', 'Multi-domain Success(%)']
Adding a policy distillation alone brings the slightest improvement to the original model. The output distillation has the highest informing rate in all guidance methods. By adding both a policy distillation and an output distillation, the model succeeded more often on the dialogue but failed on informing rate, which indicates the decision helps less on entity suggestion and has more effect on properties informing, which involves more decision makings on multiple turns. We also proofed that using a single universal model as the teacher is not well as using individual teachers, for the informing rate of the universal distillation is lower than the output distillation one.