Resolving Language and Vision Ambiguities Together: Joint Segmentation & Prepositional Attachment Resolution in Captioned Scenes
Figure 5: Visualizations for 3 different prepositions (red = high scores, blue = low scores). We can see that our model has implicitly learned spatial arrangements unlike other spatial relation learning (SRL) works.
['Feature set', 'PASCAL-50S Instance-Level Jaccard Index', 'PASCAL-50S PPAR Acc.', 'PASCAL-Context-50S PPAR Acc.']
Visualizing Prepositions. These visualizations show the score obtained by taking the dot product of distance features (Euclidean and directional) between object1 and object2 connected by the preposition with the corresponding learned weights of the model, considering object2 to be at the center of the visualization. These were simply recovered as an intermediate step towards reranking SS + PPAR hypotheses. In addition, our approach is more general.