Ensemble-Based Deep Reinforcement Learning for Chatbots
Table 5: Automatic evaluation of chatbots on test data
['Agent/Metric', 'Dialogue Reward', 'F1 Score', 'Recall@1']
As expected, the Upper Bound agent achieved the best scores and the Lower Bound agent the lowest scores. The difference in performance between the Ensemble agent and the Seq2Seq agent is significant at p=0.0332 for the Fluency metric and at p<0.01 for the other metrics (Engagingness and Consistency)—based on a two-tailed Wilcoxon Signed Rank Test.