Memory-augmented Dialogue Management for Task-oriented Dialogue Systems
Table 8. Fine-grained and overall accuracy on the ALDM dataset. The number in bracket is the accuracy at the session level, and the number without bracket is at the turn level.
['Metrics MEM', 'DA type 64.9', 'DA type (1.4)', 'Slot-value 73.5', 'Slot-value (0.0)', 'Mask 100.0', 'Mask (100.0)', 'All 0.0', 'All (0.0)']
Although the average number of the slot values in ALDM is much larger than that in the other two dataset, we still obtain high slot-value accuracy. This can be attributed to the high data quality of ALDM which is carefully cleaned before training. By removing the slot-value memory (RNN and MAD-SM) we can see that the slot-value accuracy decreases remarkably, which shows the ability of slot-value memory for maintaining dialogue states. This is because of the nature of the ALDM dataset that the user responses are mainly one-word sentences, which makes no difference between the models with/without attention mechanism.