MAttNet: Modular Attention Network for Referring Expression Comprehension
Table 1: Comparison with state-of-the-art approaches on ground-truth MS COCO regions.
['[EMPTY]', '[EMPTY]', 'feature', 'RefCOCO val', 'RefCOCO testA', 'RefCOCO testB', 'RefCOCO+ val', 'RefCOCO+ testA', 'RefCOCO+ testB', 'RefCOCOg val*', 'RefCOCOg val', 'RefCOCOg test']
First, we compare our model with previous methods using COCO’s ground-truth object bounding boxes as proposals. Results are shown in Table. As all of the previous methods (Line 1-8) used a 16-layer VGGNet (vgg16) as the feature extractor, we run our experiments using the same feature for fair comparison. Note the flat fc7 is a single 4096-dimensional feature which prevents us from using the phrase-guided attentional pooling in Fig. Despite this, our results (Line 9) still outperform all previous state-of-the-art methods. After switching to the res101-based Faster R-CNN (res101-frcn) representation, the comprehension accuracy further improves another ∼3% (Line 10). Note our Faster R-CNN is pre-trained on COCO’s training images, excluding those in RefCOCO, RefCOCO+, and RefCOCOg’s validation+testing. Our full model (Line 11) with phrase-guided attentional pooling achieves the highest accuracy over all others by a large margin.