Multimodal Sentiment Analysis: Addressing Key Issues and Setting up the Baselines
TABLE II: Accuracy reported for speaker-exclusive (Sp-Ex) and speaker-inclusive (Sp-In) split for Concatenation-Based Fusion. IEMOCAP: 10-fold speaker-exclusive average. MOUD: 5-fold speaker-exclusive average. MOSI: 5-fold speaker-exclusive average. Legend: A stands for Audio, V for Video, T for Text.
['Modality Combination', 'IEMOCAP Sp-In', 'IEMOCAP Sp-Ex', 'MOUD Sp-In', 'MOUD Sp-Ex', 'MOSI Sp-In', 'MOSI Sp-Ex']
IEMOCAP: As this dataset contains 10 speakers, we performed a 10-fold speaker-exclusive test, where in each round exactly one of the speakers was included in the test set and missing from train set. The same SVM model was used as before and accuracy was used as performance metric. MOUD: This dataset contains videos of about 80 people reviewing various products in Spanish. Each utterance in the video has been labeled as positive, negative, or neutral. In our experiments, we consider only samples with positive and negative sentiment labels. The speakers were partitioned into 5 groups and a 5-fold person-exclusive experiment was performed, where in every fold one out of the five group was in the test set. MOSI: MOSI dataset is rich in sentimental expressions, where 93 people review various products in English. The videos are segmented into clips, where each clip is assigned a sentiment score between −3 to +3 by five annotators. We took the average of these labels as the sentiment polarity and naturally considered two classes (positive and negative). Like MOUD, speakers were divided into five groups and a 5-fold person-exclusive experiment was run. For each fold, on average 75 people were in the training set and the remaining in the test set. The training set was further partitioned and shuffled into 80%–20% split to generate train and validation sets for parameter tuning.