DialoGPT : Large-Scale Generative Pre-trainingfor Conversational Response Generation
Table 8: Human evaluation significance test. Bold results represent differences that are NOT statistically significant. Notation: 1 - Human response; 2 - DialoGPT 345M; 3 - PersonalityChat; 4 - DialoGPT 345M w/ MMI; 5 - DialoGPT 345M Beam search; 6 - DialoGPT 762M
['[EMPTY]', 'System 1 Mean', 'Std', '95% CI', 'System 2 Mean', 'Std', '95% CI', 'Pairwise Std', '95% CI', 'P-Value']
Significance testing for the difference in means was performed using 10K bootstrap iterations. P-values are computed at α=0.05. The differences between 345M (2) and 762M (6) models are not significant. Notably also, the differences between 345M model (2) and human response (1) are not statistically significant.