Predicting the Popularity of Micro-videos with Multimodal Variational Encoder-Decoder Framework
Table 4: Influence of modality missing problems to MMVED-TMP model under nMSE and SRC criteria.
['Available Modalities', 'nMSE-TMP', 'SRC']
In the experiments, we fix the weights of the trained MMVED-TMP model, eliminate one modality with the corresponding encoder network at a time, and report the test performance. Aside from the strong temporal features, such results could also be explained by the PoE nature of the multimodal encoder, where the decision for which information to encode from the micro-video content into the hidden representation is made based on the complementary information from all modalities, weighted by their confidence. The elimination of visual and aural modality, on the other hand, shows least effects on the performance, since although features from both modalities have the largest dimension, they are redundant and can be complemented by information from other available modalities, whereas the user attributes from the social modality are compact and comparatively irreplaceable.