Dialog State Tracking: A Neural Reading Comprehension Approach
Table 7: Analyzing the overall model robustness for conversation depth for MultiWOZ-2.0
['Conversation', 'Total', '% Incorrect']
More precisely, we segment a given set of dialogs into individual turns and measure the state accuracy for each of these segments. We mark a turn correct only if all the slots in its state are predicted correctly. We observe that the model performance degrades as the number of turns increase. The primary reason for this behavior is that an error committed earlier in the conversation can be carried over for later turns. This results in a strictly higher probability for a later turn to be incorrect as compared to the turns earlier in the conversation. These results motivate future research in formulating models for state tracking that are more robust to the depth of the conversation.