The Eighth Dialog System Technology Challenge
Table 7: Evaluation Results for Schema-Guided State Tracking track
['Team', 'Joint Goal Accuracy', 'Avg Goal Accuracy', 'Active Intent Accuracy', 'Requested Slots F1']
We received submissions from 25 teams. The evaluation set includes three new domains - “Messaging", “Payment" and “Trains", in addition to having a few unseen APIs for some of the domains present in training and dev sets. We observe that the submitted models are able to generalize well to new APIs and domains. Most of the submitted models make use of pre-trained models like BERT Devlin et al. to generalize to unseen domains and APIs.