Cross-lingual and Multilingual Speech Emotion Recognition on English and French
Table 2: Results as unweighted average recall (UAR), cross-lingual: only trained on source language, CL + FT: pre-trained on source language and fine-tuned on 500 samples from target lanugage (CL - cross-lingual, FT - fine-tuning).
['[EMPTY]', 'IEMOCAP (English)', 'IEMOCAP (English)', 'Recola (French)', 'Recola (French)']
The performance measure used throughout all experiments is unweighted average recall (UAR, i.e. the average of the recall for each class). This measure best reflects the overall accuracy when the dataset is imbalanced with respect to the number of samples per class. Cross-lingual training is useful in cases where no or only little training data in the target language is available. We therefore examine the performance of the system when trained on one language and tested on the other (and vice versa), given the same type of speech (human-human interaction). For arousal, the performance drops notably for IEMOCAP (trained on Recola) compared to the mono-lingual baseline, achieving 59.32% UAR. For Recola (trained on IEMOCAP) it remains stable (60.77% mono-lingual, 61.27% cross-lingual). For valence, both results are below chance, suggesting that valence prediction might be more language-dependent than predicting arousal.