On Enhancing Speech Emotion Recognition using Generative Adversarial Networks
Table 1: Classification results of a SVM trained using different set-ups involving the synthetically generated 2-D representation of the real 1582-D openSMILE and the synthetically generated higher dimensions samples with and without real data. Feature vectors of same dimensionality were used in the test set
['Dataset', 'UAR (%)']
In this experiment, we perform a leave one session out cross-validation experiment on the IEMOCAP dataset. Given that each session contains a unique pair of participants, this evaluation is also speaker independent. We train the vanilla/conditional GAN on four IEMOCAP sessions and generate synthetic samples. We train a Support Vector Machine (SVM) model with radial basis function as kernel, SVMvan, on the 2-dimensional projection space learned by the adversarial auto-encoder and train it under three conditions: (i) using only the synthetic samples generated by the vanilla GAN, (ii) using only the real samples in the four training sessions and, (iii) using a combination of both synthetic and real samples. The trained models are evaluated on the 2-dimensions representations of the test set, as yielded by the adversarial auto-encoder. Similarly, after obtaining samples from the conditional GAN, we train another SVM model, SVMcon, on the 1582-dimensional openSMILE feature space. We again perform the three sets of experiments as mentioned above. SVMcon is evaluated on the test partition in the 1582-dimensional openSMILE feature space. It is clearly evident that by using only the synthetically generated samples for training the SVM we beat the chance accuracy by a big margin. It is worth noting that in case of simple GAN, the SVM’s performance trained with only synthetic data is comparable to that of a SVM trained with actual 2D code vectors. This could probably be because the 2D code vectors follow a specific distribution enforced by the adversarial auto-encoder framework and not just any random distribution. The specific distribution being the mixture of four Gaussian components is not as complex as real world distributions and hence the GAN model could easily learn that distribution. that’s not the case when appending them with synthetic data points generated from improved conditional GAN. An improvement in UAR in both cases shows the potential of using synthetically generated data along with real data for training and classification purposes.