Guided Dialog Policy Learning: Reward Estimation for Multi-Domain Task-Oriented Dialog
Table 6: The count of human preference on dialog session pairs that GDPL wins (W), draws with (D) or loses to (L) other methods based on different criteria. One method wins the other if the majority prefer the former one.
['VS.', 'Efficiency W', 'Efficiency D', 'Efficiency L', 'Quality W', 'Quality D', 'Quality L', 'Success W', 'Success D', 'Success L']
Table 6 presents the results of human evaluation. GDPL outperforms three baselines significantly in all aspects (sign test, p-value < 0.01) except for the quality compared with ACER. Among all the baselines, GDPL obtains the most preference against PPO.