Dialogue Act Sequence Labeling using Hierarchical encoder with CRF
Table 9: WE is Word Embedding layer, UL is Utterance Layer, CL is Conversation Layer, LR is Logistic regression and CRF is Conditional Random Field.
['[BOLD] Model', '[BOLD] Accuracy  [BOLD] with LR', '[BOLD] Accuracy  [BOLD] with CRF']
From the table, we observe that the models WE, WE+UL, and WE+UL+CL with LR layer at the top produce an accuracy of 71.4%, 72.2%, and 74.1%, respectively. In the final layer, if LR is replaced with CRF then the accuracy of WE, WE+UL, and WE+UL+CL (Hierarchical Bi-LSTM-CRF) is 72.2%, 72.7%, and 79.2%, respectively. From these results it is clear that adding additional layers, viz. utterance layer and conversation layer, improve the results by a few notches. Also, replacing LR with CRF further improves the results. Note that the accuracy of WE+UL with LR and WE with CRF is same. We understand that the output of utterance layer at each time step is a vector representing the context of the utterance till that word. The word vector at the last time step is the final representation of the utterance. This means, adding an utterance layer generates a compositional vector of all words in an utterance, and thus serves as a good representation of all words in the utterance. Adding the utterance layer and replacing the LR with CRF in the existing model produces more or less the same result. Addition of conversation layer results in major improvement in the accuracy, approximately 2% absolute points with LR in the final layer, and 6% absolute points with CRF . This is because the output of conversation layer for an utterance is a representational vector capturing the context of itself and utterances preceding it.