Augmenting Non-Collaborative Dialog Systems with Explicit Semantic and Strategic Dialog History
Table 2: Human evaluation ratings for (on a scale from 1 to 5) for FeHED, FeHED−FST-DA, FeHED−FST-S, and HED. We conducted third-person rating and second-person rating. Sale price is normalized.
['Second-person Rating  [BOLD] Models', 'Second-person Rating Persuasive', 'Second-person Rating Coherent', 'Second-person Rating Natural', 'Second-person Rating Sale Price', 'Third-person Rating Persuasive', 'Third-person Rating Coherent', 'Third-person Rating Natural', 'Third-person Rating Sale Price']
For third-person rating, we asked an expert to generate 20 dialogs by negotiating with FeHED, FeHED−FST-DA, FedHED−S, HED+RNN and HED respectively (5 dialogs each). We then recruited 50 people on AMT to rate these generated dialogs. Result shows that FeHED is more persuasive, coherent and natural than all the baselines. For second-person rating, we asked 50 participants on AMT to play the role of buyer to negotiate with FeHED, FeHED−FST-DA, FeHED−FST-S, HED+RNN and HED respectively. Results show that FeHED outperforms RNN and other models on all the metrics except naturalness. It is likely because RNN is trained jointly with HED, but it is not good at explicitly tracking and preserving the history. Although FST is learned separately, it forces the model to learn the history through a list of traversed states. We analyze the generated dialogs for human evaluation and find that baselines are more likely to accept unfair offers and apply inappropriate strategies.