Multimodal Language Analysis with Recurrent Multistage Fusion
Table 4: Effect of varying the number of stages on CMU-MOSI sentiment analysis performance. Multistage fusion improves performance as compared to single stage fusion.
['Dataset', '[BOLD] CMU-MOSI', '[BOLD] CMU-MOSI', '[BOLD] CMU-MOSI', '[BOLD] CMU-MOSI', '[BOLD] CMU-MOSI']
Q1: To study the effectiveness of the multistage fusion process, we test the baseline RMFN-R1 which performs fusion in only one stage instead of across multiple stages. This model makes the strong assumption that all cross-modal interactions can be modeled during only one stage. Q2: We test baselines RMFN-RK which perform K stages of fusion. Further increases led to decreases in performance and we hypothesize this is due to overfitting on the dataset.