Learning spectro-temporal features with 3D CNNs for speech emotion recognition
TABLE V: Confusion matrix (%). N: Neutral, H: Happy, S: Sad, A: Angry.
['Method DNN-ELM', 'Method DNN-ELM', 'Method DNN-ELM', 'Method DNN-ELM', 'Method DNN-ELM', 'Method LSTM-ELM', 'Method LSTM-ELM', 'Method LSTM-ELM', 'Method LSTM-ELM', 'Method LSTM-ELM']
Next, we investigate how each class becomes discriminative in the feature space. To this end, we visualise utterance-level representations of one test set of our cross-validation by using t-distributed stochastic neighbour embedding (T-SNE) T-SNE is a non-linear transform technique to embed high-dimensional data into a space of two or three dimensions. We obtain utterance-level features from the top fully-connected layers (before the softmax layer). Compared to 2D-CNN-LSTM-DNN, 3D-CNN-DNN shows a more discriminative separation between “neutral” and “anger”. The gains are 8 and 28%, respectively. Classification of “neutral” and “sadness” shows the similar result too. The gain for “sadness” is 17%.