The Bottleneck Simulator: A Model-based Deep Reinforcement Learning Approach
Table 4: First A/B testing experiment topical specificity and coherence by policy. The columns are average number of noun phrases per system utterance (System NPs), average number of overlapping words between the user’s utterance and the system’s response (This Turn), and average number of overlapping words between the user’s utterance and the system’s response in the next turn (Next Turn). Stop words are excluded. 95% confidence intervals are also shown.
['[BOLD] Policy', '[BOLD] System NPs', '[BOLD] Word Overlap  [BOLD] This Turn', '[BOLD] Word Overlap  [BOLD] Next Turn']
This indicates that the Bottleneck Simulator has the most coherent and engaging dialogues out of all the evaluated policies. This is in agreement with its excellent performance w.r.t. real-world user scores and w.r.t. the preliminary evaluation. A further analysis of the selected responses indicates that the Bottleneck Simulator has learned a more risk tolerant strategy.