Multimodal Logical Inference System for Visual-Textual Entailment
Table 5: Experimental results on Visual Genome (F1). “#” stands for the number of query sentences categorized into that phenomenon.
['Phenomena (#) Con', 'Phenomena (#) (17)', 'Simple 36.40', 'Hybrid 41.66']
Hybrid shows better performance for all phenomena than Simple one, improving by 19.59% on Q, 9.97% on Rel and 7.01% on Neg, over Simple, suggesting that the proposed complex translation is useful for inference using semantically complex sentences including quantifier and negation. Every green tree is tall and (b) No umbrella is colorful, each containing universal quantifier and negation, respectively. Our system successfully performs inference on these queries, returning the correct images, while excluding wrong ones (note that the third picture in (a) contains short trees).