Teacher-Student Framework Enhanced Multi-domain Dialogue Generation
Table 2: The result between different output distillation methods.
['distill', 'Multi-domain BLEU', 'Multi-domain Inform(%)', 'Multi-domain Success(%)']
Top-128 brings the best performance for the model. It is well explainable that the top-k distillation removes the unnecessary information the teachers give to the student. Meanwhile, the significant words are kept in the distillation to ensure the response is good enough. As the size of the distillation reduced, both the success rate and the informing rate decrease due to the loss of enough guidance. The result shows slightly abnormal when k=1. We believe that top-1 distillation is in the same form as to add more grounding truth to the training data.