MA-DST: Multi-Attention-Based Scalable Dialog State Tracking
Table 5: F1 Score and Average Slot Accuracy Domain Wise
['Domain Level Statistics Domain', 'Domain Level Statistics F1-Score', 'Domain Level Statistics Slot Acc.']
In this section we analyze the errors being made by the model on MultiWoz 2.1 dataset. In terms of F1-Score, the model performs worse for Taxi domain. The average slot accuracy for Taxi domain is high because a vast number of taxi domain’s slots are ‘‘none" (i.e. not present in the dialog), which model easily identifies. These slots are difficult to predict for the model because, unlike categorical slots, these slots can take on a large number of possible values and are more likely to encounter unseen values. On the other end of spectrum, we have slots like restaurant-bookday, hotel-bookstay, hotel-bookday, and train-day, for which the model is able to achieve more than 99% in terms of average slot accuracy. As expected, most of the top-perfoming slots are categorical, i.e. they can take only a small number of different values from a pre-defined set.