Learning Robust Dialog Policies in Noisy Environments
Table 6: One dialog turn between a simulated user and Fixed Policy (MovieBot - Execute/Confirm/Elicit) agent, and Dueling DDQN agent, along with the ground truth of the utterance and intent (first column of the table), the hypothesis from the ASR/NLU components (second column), and the confidence scores of the hypotheses with a range between 0 to 1 (third column). The DDQN agent can decide not to clarify based on the combination of specific intent and confidence scores.
['reference', 'hypothesis', 'confidence score']
To gain insight into the learned policy’s properties, we conducted a qualitative analysis to compare 100 dialogs generated by Dueling DDQN and Fixed Policy agents. To achieve a fair comparison, we fixed the random number generator so that the user simulator generated identical sequences of intents and utterances to interact with both agents. Overall, 28% of the dialogs were identical. Considering the remaining dialogs, two main reasons for the differences can be identified. First, the DDQN agent can decide not to clarify based on the combination of specific intent and confidence scores. In this example, since the ASR score is relatively low (0.239), the Fixed Policy agent decides to ask a clarifying question (denoted as  ). However, the low ASR score comes from the misrecognition of words which have small effect on the recognition of the intent as indicated by the high NLU confidence score. As such, the DDQN agent learns that the ASR confidence score is less important in this case and the agent executes the command and provides the requested information. Secondly, the DDQN agent learns that elicitation brings more information than confirmation since the user simulator repeats the original utterance. 82% of the clarifying questions asked by the DDQN agent are elicitation, as opposed to 61% for the Fixed Policy.