An End-to-End Dialogue State Tracking System withMachine Reading Comprehension and Wide & Deep Classification
Table 5: Results of WD-DST in development dataset
['Accuracy', 'Average', 'Boolean Slot', 'Text-based Slot']
We compare the results between the Wide & Deep model and the RoBERTa-based classification model to idenfy the effect of hand-crafted features. The RoBERTa-based classification model achieves an average accuracy of 0.9355 on the development dataset, while the Wide & Deep model achieves 0.9751. Contrast with the single deep learning model, Wide & Deep model can enhance the system performance greatly. This is because the input of the Wide & Deep model adds a lot of hand-crafted semantic features, which is more effective than using RoBERTa model only. The combination of deep semantic understanding features and shallow linguistic features is the most important factor for boolean and text-based slot tracking tasks. Moreover, there is a problem with models that only use RoBERTa, if the dialogue history is too long, it will truncate some key information which resulting in a decrease of the model’s accuracy.