Low-Resource Response Generation with Template Prior
Table 3: Human annotation results. W, L, and T refer to Win, Lose and Tie respectively. The first three rows are results on question response generation, and the last three rows are results on sentiment response generation. The ratios are calculated by combining labels from the three judges.
['Models', 'Fluency W(%)', 'Fluency L(%)', 'Fluency T(%)', 'Relevance W(%)', 'Relevance L(%)', 'Relevance T(%)', 'Richness W(%)', 'Richness L(%)', 'Richness T(%)', 'Kappa']
In terms of all the three aspects, S2S-Temp is better than all the baseline models. The values of kappa are all above 0.6, indicating substantial agreement among the annotators. When the size of paired data is small, the basic Seq2Seq model tends to generate more generic responses. That is why the gap between S2S-Temp and Seq2Seq is much smaller on fluency than those on the other two aspects. With the latent variables, CVAE brings both content and noise into responses. Therefore, the gap between S2S-Temp and CVAE is more significant on fluency and relevance than that on richness. HTD can greatly enrich the content of responses, which is consistent with the results in Wang et al. ECM does not perform well on both automatic evaluation and human judgement.