Towards Conversational Recommendation over Multi-Type Dialogs
Table 5: Human evaluation results at the level of turns and dialogs.
['Methods↓ Metrics→', 'Turn-level results Fluency', 'Turn-level results Appro.', 'Turn-level results Infor.', 'Turn-level results Proactivity', 'Dialog-level results Goal success rate', 'Dialog-level results Coherence']
Results All human evaluations are conducted by three persons. In particular, S2S tends to generate safe and uninformative responses, failing to complete goals in most of dialogs. Our two systems can produce more appropriate and informative responses to achieve higher goal success rate with the full use of goal information and knowledge. Moreover, the retrieval-based model performs better in terms of fluency since its response is selected from the original human utterances, not automatically generated. But it performs worse on all the other metrics when compared to the generation-based model. It might be caused by the limited number of retrieval candidates. Finally, it can be seen that there is still much room for performance improvement in terms of appropriateness and goal success rate, which will be left as the future work.