ACL’2020 Cross-Modality Relevance for Reasoning on Language and Vision
Table 1: Accuracy on NLVR2.
['[BOLD] Models', 'Dev%', 'Test%']
Transformer based models (VisualBERT, LXMERT, and CMR) outperform other models (N2NMN Hu et al. by a large margin. This is due to the strong pre-trained single-modality representations and the Transformers’ ability to reshape the representations that align the spaces. Furthermore, CMR shows the best performance compared to all Transformer-based baseline methods and achieves state-of-the-art. VisualBERT and CMR have similar cross-modality alignment approach. CMR outperforms VisualBERT by 12.4%. The gain mainly comes from entity relevance and relational relevance that model the relations.