The Influence of Context on Dialog Act Recognition
Table 16: Accuracy (%) results obtained on the Tilburg DialogBank English dialogs using context information extracted from the n preceding segments in different forms. The first two rows refer to context information provided in the form of n-grams while the remaining two refer to context information provided in the form of dialog act classifications relative to the Task dimension only or all the dimensions.
['[EMPTY]', '# Previous Segments 0', '# Previous Segments 1', '# Previous Segments 2', '# Previous Segments 3', '# Previous Segments 4', '# Previous Segments 5']
Concerning data obtained from the Tilburg DialogBank, we performed experiments on both English and Dutch dialogs in order to assess possible language-independent results. We can see that, in this case, context information in the form of untagged n-grams led to much more irregular results than in the case of the Switchboard corpus. Accuracy insignificantly decreases 0.80 percentage points below the baseline when using a single preceding segment but beyond that it starts increasing, from 0.45 percentage points beyond the baseline when using 2 preceding segments up to a significant 3.43 when using 5 preceding segments. For the remaining approaches, the first preceding segment is still the one that leads to the largest accuracy boost. In this sense, the pattern produced by index-tagged n-grams is a mix between the ones produced on the Switchboard corpus. The first preceding segment significantly improves accuracy by 3.71 percentage points and, beyond that, accuracy starts decreasing but never below the baseline. However, these differences are all statistically insignificant. Nonetheless, the 0.28 percentage point difference between the best results obtained using untagged n-grams and tagged n-grams is statistically significant. Finally, the results obtained using dialog act annotations reveal the same influence patterns as on the Switchboard corpus, with a noticeable decrease of influence in relation to the distance to the segment being classified. Furthermore, this approach was also the one that performed better, with a 17.54 percentage point improvement over the baseline, versus the 3.71 percentage points of index-tagged n-grams. Information provided by dimensions other than Task was able to improve accuracy, but only by 0.25 percentage points. However, if we consider a single preceding segment, the improvement is of 2.18 percentage points, which is more pronounced. Nonetheless, in both cases, the result difference is statistically insignificant.