A Deep Reinforcement Learning Chatbot
Table 9: First A/B testing experiment topical specificity and coherence of the six different policies. The columns are average number of noun phrases per user utterance (User NPs), average number of noun phrases per system utterance (System NPs), average number of overlapping words between the user’s utterance and the system’s response (Word overlap t→t+1), and average number of overlapping words between the user’s utterance and the system’s response in the next turn (Word overlap t→t+3). 95% confidence intervals are also shown. Stop words are excluded.
['[BOLD] Policy', '[BOLD] User NPs', '[BOLD] System NPs', '[BOLD] Word overlap  [ITALIC] t→ [ITALIC] t+1', '[BOLD] Word overlap  [ITALIC] t→ [ITALIC] t+3']
For topic specificity, we measure the average number of noun phrases per user utterance and the average number of noun phrases per system utterance. The more topic specific the user is, the higher we would expect the first metric to be. Similarly, the more topic specific the system is the higher we would expect the second metric to be. For topic coherence, we measure the word overlap between the user’s utterance and the system’s response, as well as word overlap between the user’s utterance and the system’s response at the next turn. The more the policy prefers to stay on topic, the higher we would expect these two metrics to be.