EDA: Enriching Emotional Dialogue Acts using an Ensemble of Neural Annotators
Table 1: Baseline validation with the SwDA test dataset. SC: Spearman Correlation between prediction of model and ground truth.
['Models', 'Accuracy', 'SC']
Utt-level-1 Dialogue Act Neural Annotator (DANA) is an utterance-level classifier that uses word embeddings ( w) as an input to an RNN layer, attention mechanism (att) formulated as: dat=softmax(att(RNN(wt,wt−1,...,wt−m))) (1) such that attention mechanism provides: n∑n=0at−n=1 (2) Finally, the softmax function is used to compute the probability distribution, which is formulated as: dat=softmax(att(RNN(ut,ut−1,ut−2))) (3) where ut is derived from RNN(wt,wt−1,...,wt−m). This model achieved 77.55% accuracy on the SwDA corpus test set see Similar to the previous model, it computes the probability of dialogue acts using the softmax function. Context-2-DANA is another context model that uses three utterances similar to the Context-1-DANA model. However, the utterances are composed of the mean of the word embeddings over each utterance, similar to the Utt-level-2-DANA model ( Hence, the Context-2-DANA model is composed of one RNN layer with three input vectors, finally topped with the softmax function for computing the probability distribution of the dialogue acts. Context-3-DANA is a context model that uses three utterances similar to the previous context models. Hence, the Context-3-DANA model combines features of almost all the previous four models to provide the recognition of the dialogue acts. As a baseline to verify the ensemble logic, we use the SwDA test dataset where we know the ground truth labels. The ensemble model logic is configured in a way that it achieves an accuracy similar to or better than one of the neural annotators. It is shown that the ensemble annotator performs well on the state of the art test data. These results are also supported by the correlation scores of the respective models. Hence, the configuration for the ensemble model that achieved the accuracy for the SwDA test dataset is explained in the following paragraph.