Evaluating Multimodal Representations on Visual Semantic Textual Similarity
Table 5: Supervised scenario: Train, validation and test results of the unsupervised models
['[BOLD] Model', '[BOLD] Modality', '[BOLD] train  [ITALIC] ρ', '[BOLD] dev  [ITALIC] ρ', '[BOLD] test  [ITALIC] ρ']
The supervised scenario. Overall, models that use a conjunction of multimodal features significantly outperform unimodal models, and this confirms, in a more competitive scenario, that adding visual information helps learning easier the STS task. The gain of multimodal models is considerable compared to the text-only models. The most significant gain is obtained when glove features are combined with resnet. The model improves more than 15.0 points. In this case, the improvement over bert is lower, but still considerable with more than 4.0 points.