Interpretable Multimodal Routing for Human Multimodal Language
Table 2: CMU-MOSEI emotion recognition. Multimodal Routing∗ denotes our method without iterative routing. We make our results bold if it is the best or close to the best (<1%).
['Models', 'CMU-MOSEI Emotion Happy', 'CMU-MOSEI Emotion Happy', 'CMU-MOSEI Emotion Sad', 'CMU-MOSEI Emotion Sad', 'CMU-MOSEI Emotion Angry', 'CMU-MOSEI Emotion Angry', 'CMU-MOSEI Emotion Fear', 'CMU-MOSEI Emotion Fear', 'CMU-MOSEI Emotion Disgust', 'CMU-MOSEI Emotion Disgust', 'CMU-MOSEI Emotion Surprise', 'CMU-MOSEI Emotion Surprise']
CMU-MOSEI emotion. We do not report RAVEN Wang et al. Compared with all the baselines, Multimodal Routing performs the best on most of the results metrics. We note that the distribution of labels is skewed (e.g., there are disproportionately very few samples labeled as “surprise”). Hence, this skewness somehow results in the fact that the model always predicts not having emotion “surprise”, thus the same accuracy for “surprise” across different approaches.