Dialogue Act Sequence Labeling using Hierarchical encoder with CRF
Table 10: Accuracy obtained using two extensions to the Hierarchical Bi-LSTM-CRF model.
['[BOLD] Extension POS', '[BOLD] Extension POS', '[BOLD] Accuracy(%) 77.9']
In our model, after obtaining utterance vectors from the conversation layer, a normalized attention weight vector is computed for each utterance vector, by computing its similarity from previous utterance vectors. These attention weights are then used to compute the context vector by taking a weighted sum of the previous K utterance vectors. The new context vector is concatenated to the utterance vector produced by the conversation layer to obtain new utterance vector, which is input to the classification layer. We experimented with this attention by varying the length of the context (number of previous utterances) i.e. K∈(10,5,3). In a conversation, an utterance at time step t is mostly dependent upon the previous two or three utterances.