Multimodal Affective Analysis Using Hierarchical Attention Strategy with Word-Level Alignment
Table 2: Accuracy (%) and F1 score on text only (T), audio only (A), and multi-modality using FAF (T+A).
['Modality', 'MOSI WA', 'MOSI F1', 'IEMOCAP WA', 'IEMOCAP F1']
However, on IEMOCAP, audio-only outperforms text-only, but as expected, there is a significant performance improvement by combining textual and audio. The difference in modality performance might because of the more significant role vocal delivery plays in emotional expression than in sentimental expression.