Speaker-change Aware CRF for Dialogue Act Classification
Table 3: Precison, Recall, and F1 score (%) of our model vs. base model on the sd and sv labels.
['Our', 'sd', 'P 80.49', 'R 86.36', 'F1 83.32']
Although our model achieves significant gains on a majority of the most frequent labels, it decreases performance for the most frequent label, sd, which accounts for 36.9% of all labels, as shown by Fig. This explains why, in terms of overall accuracy, our improvements are modest. We can observe that, as opposed to the base model, our model has lower sd and higher sv recall values. A similar observation can be made for precision scores. Thus, the prediction between sd and sv is a trade-off made by models. It is also interesting to note that our model is superior for both labels in terms of F1 score.