Scalable and Accurate Dialogue State Tracking via Hierarchical Sequence Generation
Table 5: The ablation study on the MultiWoZ dataset with the joint domain accuracy (JD Acc.), joint domain-slot accuracy (JDS Acc.) and joint goal accuracy (JG Acc.) on the test set. For “- moveDrop”, we move the dropout layer to be in front of the final linear layer before the Softmax. For “- postprocess”, we further fix the decoder embedding layer and remove the post-processing during model evaluation. For “- ShareParam”, we further remove the parameter sharing mechanism on the encoders and the attention modules. For “- Order”, we further arrange the order of the slots according to its global frequencies in the training set instead of the local frequencies given the domain it belongs to. For “- Nested”, we do not generate domain sequences but generate combined slot sequences which combines the domain and the slot together. For “- BlockGrad”, we further remove the gradient blocking mechanism in the CMR decoder.
['[BOLD] Model', '[BOLD] JD Acc.', '[BOLD] JDS Acc.', '[BOLD] JG Acc.']
Table 5: The ablation study on the MultiWoZ dataset with the joint domain accuracy (JD Acc.), joint domain-slot accuracy (JDS Acc.) and joint goal accuracy (JG Acc.) on the test set. [CONTINUE] From Table 5, We can further calculate that given the correct slot prediction, COMER has 87.42% chance to make the correct value prediction. While COMER has done great job on domain prediction (95.52%) and value prediction (87.42%), the accuracy of the slot prediction given the correct domain is only 58.43%. [CONTINUE] We can also see that the JDS Acc. has an absolute boost of 5.48% when we switch from the combined slot representation to the nested tuple representation.