Dialogue Act Sequence Labeling using Hierarchical encoder with CRF
Table 3: Hyperparameter tuning – the 2nd column lists the various values tried, while the 3rd column lists the final value chosen for the corresponding hyperparameter.
['[BOLD] Parameter', '[BOLD] Range', '[BOLD] Final']
Conversations with the same number of utterances were grouped together into mini-batches, and each utterance in a mini-batch was padded to the maximum length for that batch. The maximum batch-size allowed was 64. We used L2 regularization of 1e−4 in the form of weight decay and the Adadelta optimizer. All other hyper-parameters were selected by tuning one hyper-parameter at a time while keeping the others fixed. The hyper-parameters were tuned using the SwDA validation set. The final set of hyper-parameters were then used to train two different models, one each on SwDA and MRDA training datasets. Dropout was applied to the embeddings obtained from the output of each encoder. The learning rate was initialized to 1.0 and reduced by a factor of 0.5 every 5 epochs. Early stopping is also used on the validation set with a patience of 5 epochs. Increasing the number of stacked LSTM layers reduced the accuracy of the model, so we settled with only one layer.