Domain-Aware Dialogue State Tracker forMulti-Domain Dialogue Systems
Table 7: Performance on each domain in Test set. * indicates the domains that contain services not seen in training, while ** indicates domains containing one seen and one unseen service.
['[BOLD] Domain', '[BOLD] Avg. GA', '[BOLD] Joint GA']
The architecture of the proposed model is robust to variations in the service schema, thus enabling to adequately predict for any new schema that was not seen in training (unseen). Such zero-shot prediction capability of the domain-aware DST is crucial, as it enables knowledge transfer from high resource domains to similar domains with no or fewer training data. We can notice that the performance on the unseen services is comparatively low with respect to the seen services, due to the fact that unseen services contain a higher number of out-of-vocabulary (OOV) slots and values. Of all the domains in test set, the domains Alarm, Messaging, Payment and Trains do not have any service in the training data. This results in the lowest average goal accuracy for Alarm, Messaging, Payment among all domains. However, for the Trains domain, we can notice that the average goal accuracy is even higher than the accuracy on some of the seen domains. This is due to the fact that the Trains domain is similar to other domains in the training set, such as Buses and Flights. This shows that the model is able to use the learned knowledge from existing domains in training data, and effectively apply this knowledge to new similar domains without any additional training data.