Improving Referring Expression Grounding with Cross-modal Attention-guided Erasing
Table 2: Ablation study results on RefCOCOg dataset.
['CM-Att-Erase (Our proposed approach)', 'CM-Att-Erase (Our proposed approach)', 'val 80.23', 'test 80.37']
Erasing methods. Different choices of erasing methods were exploited by previous works. Since attention weights are already good indicators of feature importance, leveraging attention as a guidance for erasing is more efficient, and the attention-guided erasing approach leads to little cost in model complexity, compared with applying a separate adversarial erasing network. Effect of cross-modal erasing. We compare our cross-modal erasing approach with erasing based on self-attention weights, where we only utilize information within the same modality for generating attention weights and performing attention-guided erasing. We also experiment on only visual erasing or sentence erasing. Iterative erasing. We observe that most referring expressions are quite short. Erasing more than one key words would significantly eliminate the semantic meaning of the sentence. Likewise, erasing the visual features for more than once would also make it impossible for the model to recognize the referred object.