CLEVR-Ref+: Diagnosing Visual Reasoning with Referring Expressions
Table 3: Referring object detection and referring image segmentation results on CLEVR-Ref+. We evaluated three existing models, as well as IEP-Ref which we adapted from its VQA counterpart.
['[EMPTY]', 'Basic 0-Relate', 'Spatial Relation 1-Relate', 'Spatial Relation 2-Relate', 'Spatial Relation 3-Relate', 'Logic AND', 'Logic OR', 'Same', 'Accuracy', 'IoU']
4.2.1 Overall Evaluation Detection models are evaluated by accuracy (i.e. whether the prediction selects the correct bounding box among given candidates), where MAttNet performs favorably against SLR. Segmentation models are evaluated by Intersection over Union (IoU), where IEP-Ref performs significantly better than RMI. This suggests the importance to model compositionality within the referring expression. We now present a more detailed analysis of various aspects.