Cross-lingual Visual Verb Sense Disambiguation
Table 2: Cross-lingual verb sense disambiguation accuracy of our unimodal models and the multimodal model. We also show the performance of a random chance baseline and a majority label baseline.
['[EMPTY]', 'Chance', 'Majority', 'Text', 'Image', 'MM']
The chance and majority label baselines perform very poorly. The unimodal textual model performs better than the unimodal visual model for German verb sense disambiguation, but we find the opposite for Spanish unimodal verb sense disambiguation. However, the early fusion multimodal model outperforms the best unimodal model for both German and Spanish. This confirms that cross-lingual verb sense disambiguation benefits from multimodal supervision compared to unimodal supervision.