AvgOut: A Simple Output-Probability Measure to Eliminate Dull Responses
Table 3: Human Evaluation results for all the models we produce on Plausibility, Richness, average of the two, and scaled difference (difference between them divided by their average). Best Results are boldfaced. Note that for the last column, lower is better since we want balance between Plausibility and Content Richness. All results are pair-wise statistically significantly different with p<0.05, except between MinAvgOut and RL on Plausibility, and between MinAvgOut and MinAvgOut+RL on Average.
['[EMPTY]', 'Plaus.', 'Rich', 'Avg.', 'SclDiff.']
We can see from the table that all our models are statistically significantly better than the baseline models on both Plausibility and Content Richness, except that RL is slightly weaker on Content Richness, which agrees with the trend in automatic evaluations. Although MinAvgOut+RL model only ranks the second on average score (statistically equivalent to MinAvgOut) in human evaluation, it achieves a good balance, and it also ranks the second in automatic diversity and the first in F1 values. We thus consider it to be our best model.