Fine-Tuning BERT for Schema-Guided Zero-Shot Dialogue State Tracking
Table 6: Evaluation results on the dev set. The “all APIs”, “seen APIs”, “and unseen APIs” represents the whole dev set, subset whose APIs appear in traning set, and subset whose APIs are unseen in training set.
['[EMPTY]', '[BOLD] Active Intent Acc.', '[BOLD] Requested Slot F1', '[BOLD] Average Goal Acc.', '[BOLD] Joint Goal Acc.']
, we can know that our system has significant poorer performance on the “unseen APIs” subset than that on “seen APIs” subset. Generally, zero-shot settings is a tough testing bed for neural network models, and it’s actually an economy way to improve the models’ performance by including a small number of unseen labelled samples into the training process, which is called few-shot learning. In order to do this, we simply random sample some dialogues in the dev set and adding them into the training process , then we evaluate the model on the remaining dialogues in the dev set.