Unsupervised Evaluation of Interactive Dialog with DialoGPT
Table 5: Relative importance of each dialog quality for predicting the overall impression. The most important qualities for turn-level and dialog-level are in bold.
['[BOLD] Quality', '[BOLD] Importance (%)']
The FED dataset can be used to examine the relative importance of each fine-grained dialog quality by measuring its contribution to the overall impression. For both turn-level and dialog-level, a regression is trained to predict the overall score given the fine-grained qualities as input. The regression weights provide insight into the fine-grained qualities that most contribute to the overall impression as labeled by human annotators. A softmax is computed over the regression weights to determine the relative contribution of each fine-grained dialog quality. A dialog quality with a higher weight contributes more to the human’s overall impression.