Dialog Policy Learning for Joint Clarification and Active Learning Queries
Table 2: Unabridged results from the final batch of the test phase. ∗ indicates the conditions whose performance is comparable to the best condition (in bold).
['Decision Policy Type', 'Clarification Policy Type', 'Active Learning Policy Type', 'Fraction of Successful Dialogs', 'Average Dialog Length']
We observe that the using a learned policy for all three functions, with A3C for choosing clarification and active learning queries, and Q-learning for deciding between clarification, active learning and guessing, results in a significantly more successful dialog system (according to an unpaired Welch t-test with p < 0.05) than most other conditions. However, the best fully learned policy uses significantly shorter dialogs than the other policies with a comparable success rate, thus making it overall more desirable.