Towards Coherent and Engaging Spoken Dialog Response Generation Using Automatic Conversation Evaluators
Table 3: Evaluators Correlation with Turn-level Ratings
['[BOLD] Metric', '[BOLD] Pearson Corr', '[BOLD] p-value']
We also performed a correlation study between the model predicted scores and human annotated scores (1 to 5) on 2000 utterances. 0.4 Pearson correlation with turn-level ratings). Considering the substantial individual differences in evaluating open-domain conversations, we observe that our evaluators with moderate level of correlation can be used to provide turn-level feedback for a human-chatbot conversation.