ASR-based Features for Emotion Recognition: A Transfer Learning Approach
Table 2: Means and variances of the MSE on the prediction of valence and arousal.
['[EMPTY]', 'Arousal Mean', 'Arousal Variance', 'Valence Mean', 'Valence Variance']
As expected, the results show, that for the speakers considered, the layers closer to the audio modality outperform the ones closer to the text modality in the ASR architecture for arousal prediction and vice versa for the valence prediction. On this we build a hypothesis that the arousal-related features learned are more related to the audio modality than the text and vice versa for the valence-related features. This hypothesis will be further explored in future work.