Towards Universal End-to-End Affect Recognition from Multilingual Speech by ConvNets
TABLE II: Results (percentage) on multilingual task for the four emotions analyzed (anger, sadness, happiness, anxiety). P stands for precision, R for recall, and F1 for F-score (or F1 measure).
['Method', 'English P', 'English R', 'English F1', 'Estonian P', 'Estonian R', 'Estonian F1', 'German P', 'German R', 'German F1', 'Spanish P', 'Spanish R', 'Spanish F1', 'Italian P', 'Italian R', 'Italian F1', 'Serbian P', 'Serbian R', 'Serbian F1']
They are represented in terms of precision, recall and F-score over each emotion and language. We obtained an average F-score of 67.7% (68.5% after fine-tuning the last layer) across all the languages using our raw waveform CNN trained on multiple languages. We obtained an average of 55.2% with the same model trained on a single language, 58.2% from the multilingual spectrogram baseline and 60% from the same baseline trained on single languages. Overall, this yields a relative improvement of 12.8% of the multilingual raw waveform CNN over the second best model, the spectrogram CNN trained on individual languages.