The Influence of Context on Dialog Act Recognition
Table 11: Accuracy (%) results on the second half of the Switchboard corpus obtained by classifiers without context information trained using different subsets of the corpus.
['[EMPTY]', '# Labels 42', '# Labels 43', '# Labels 44']
However, the labels used were the manual annotations of the corpus and, thus, the obtained results are an upper bound for the approach. In order to assess the performance of this approach without relying on gold standard annotations, we performed experiments using automatic classifications. To obtain the automatic classifications, we split the corpus in half and trained classifiers without context information on different subsets of the corpus and used them to predict the labels for the second half. We used three different subsets – the second half, the whole corpus, and the first half – to assess the impact of the dependence between the training and evaluation sets, from complete dependence when using the second half to train to complete independence when using the first half. As expected, the accuracy when using an independent set to train is much lower than when using a dependent set.