How to Build User Simulators to Train RL-based Dialog Systems
Table 1: Automatic metrics and human evaluation scores of different user simulators. Automatic metrics include, perplexity per word (PPL), vocabulary size (Vocab), average utterance length (Utt). Human evaluation metrics include, sentence fluency (Hu.Fl), coherent (Hu.Co), goal adherence (Hu.Go), language diversity (Hu.Div) and overall score (Hu.All).
['[BOLD] Simulators', '[BOLD] NLU', '[BOLD] DM', '[BOLD] NLG', '[BOLD] PPL', '[BOLD] Vocab', '[BOLD] Utt', '[BOLD] Hu.Fl', '[BOLD] Hu.Co', '[BOLD] Hu.Go', '[BOLD] Hu.Div', '[BOLD] Hu.All']
Perplexity (Direct) Perplexity measures the language generation quality of the user simulator. For each simulator model, we generate 200 dialogs with the third-party Rule-System and train a trigram language model with the data. Then we test the model and compute the perplexity with 5000 user utterances sampled from MultiWoz. Although the perplexity for retrieval models is the highest in both agenda-based and SL-based simulators, it also possesses the biggest vocabulary set and the longest average utterance length. Another common automatic metrics used to assess the language model is BLEU, but since this is a user simulator study and we don’t have ground truth, BLEU score is not available. Vocabulary Size (Direct) Vocabulary size is a simple and straightforward metric that measures the language diversity. As expected, retrieval-based models have the biggest vocabulary set. However, Agenda-Generation has the smallest vocabulary set. The possible reason behind is that we adopt a vanilla greedy seq2seq that suffers from generating the most frequent words. The direct evaluation of the user simulator is conducted by asking 10 volunteers to read the simulated dialogs between different simulators and the third-party Rule-System. Each of the 10 volunteers would rate five randomly-selected dialogs generated from each model, and the average of the total 50 ratings is reported as the final human-evaluation score. The Rule-System is built solely based on hand-crafted rules with no knowledge about any of the simulators, and therefore is fair to all of them. We design four metrics to assess the user simulator’s behaviour from multiple aspects. Satisfaction. Solving the user’s problem doesn’t necessarily lead to user satisfaction sometimes. It also depends on the system’s efficiency and latency. Therefore, besides Solved Ratio, we also directly ask Turkers how satisfied they are with the system. The result shows that among all systems, Sys-AgenR model received the highest score.