Learning Factorized Multimodal Representations
Table 6: Information-Based interpretation results showing ratios ri=MI(Fy,^Xi)MI(Fai,^Xi), i∈{(ℓ)anguage,(v)isual,(a)coustic} for the POM dataset for personality traits prediction.
['Ratio', '[ITALIC] rℓ (language)', '[ITALIC] rv (visual)', '[ITALIC] ra (acoustic)']
We observe that the language modality is also the most informative while the visual and acoustic modalities are almost equally informative. For example, the same sentence “this movie was great” can convey significantly different messages on speaker confidence depending on whether it was said in a loud and exciting voice, with eye contact, or powerful gesticulation.