Evaluating Prerequisite Qualities for Learning End-to-End Dialog Systems
Table 7: Ubuntu Dialog Corpus results. The evaluation is retrieval-based, similar to that of Reddit (Task 4). For each dialog, the correct answer is mixed among 10 random candidates; Hits@1 (in %) are reported. Methods with † have been ran by Lowe et al. (2015).
['Methods', '[EMPTY]', 'Validation (hits@1)', 'Test (hits@1)']
The evaluation is similar to that of Reddit (Task 4): each correct answer has to be retrieved among a set of 10, mixed with 9 randomly chosen candidate utterances. We used the same MemN2N architecture as before. all models were selected using validation accuracy. On this dataset, which has longer dialogs than those from the Movie Dialog Corpus, we can see that running more hops on the memory with the MemN2N improves performance: the 1-hop model performs similarly to the LSTM but with 2-hops and more we can gain more than a +8% increase over the previous best reported model. Using even more hops still improves over 1-hop but not much over 2-hops.