The Influence of Context on Dialog Act Recognition
Table 17: Accuracy (%) results obtained on the Tilburg DialogBank Dutch dialogs using context information extracted from the n preceding segments in different forms. The first two rows refer to context information provided in the form of n-grams while the remaining two refer to context information provided in the form of dialog act classifications relative to the Task dimension only or all the dimensions.
['[EMPTY]', '# Previous Segments 0', '# Previous Segments 1', '# Previous Segments 2', '# Previous Segments 3', '# Previous Segments 4', '# Previous Segments 5']
It shows that, as expected, accuracy results are lower than the ones obtained on English dialogs, since the amount of data is smaller. However, the importance of context information is still noticeable. Contrarily to what happened with the English dialogs, using context information in the form of untagged n-grams followed a detrimental pattern as on the Switchboard corpus. On the other hand, index-tagged n-grams improved accuracy up to the third preceding segment, obtaining the best result on this dataset with 3.68 percentage points over the baseline. However, the first preceding segment was still the most informative, leading to an accuracy improvement of 2.94 percentage points. Beyond the third preceding segment, accuracy started significantly decreasing, even to values below the baseline. Context information provided in the form of dialog act labels was less effective on this dataset, with a maximum improvement of 2.94 percentage points over the baseline. Furthermore, the results did not follow the same pattern as on the previously described experiments. In fact, there is no statistical significance in any of the differences for this approach. This suggests that providing context information in the form of dialog act labels requires larger amounts of training data to be effective in comparison to information provided in the form of index-tagged n-grams. As for information provided by other dimensions, as on the English dialogs, it was also able to slightly improve accuracy, in this case by a maximum of 1.83 percentage points. Given the irregular patterns obtained for Dutch, it is hard to draw language-independent conclusions. However, the importance of context information is noticeable in both cases, as well as the importance of representing context information in a form distinguishable from the information related to the segment being classified, that is, in a way that increases entropy. This is shown in both languages by the improvements provided by index-tagged n-grams and dialog act labels and the detrimental impact of untagged n-grams. Still, we believe that experiments using larger amounts of data in languages other than English could lead to interesting conclusions regarding the language-independence of the influence of context on dialog act recognition.