Aligning Visual Regions and Textual Concepts for Semantic-Grounded Image Representations
Table 5: Evaluation of different guiding scheme.
['Model', 'BLEU-1', 'BLEU-2', 'BLEU-3', 'BLEU-4', 'METEOR', 'ROUGE', 'CIDEr', 'SPICE']
We can either start with the textual concepts guiding the integration of the visual features or let the latter to take the initiative. Even if the role of visual features and textual concepts are equivalent in mutual attention, the choice of guiding scheme could make a difference. We examine the performance of Visual Attention and Concept Attention when the visual features first attend to the textual concepts. Especially, the performance of the Visual Attention has even been impaired. The rationale for such phenomenon is presumably the limited visual receptive field of the original visual features, which makes them inadequate to integrate the textual concepts. As to the textual concepts, they are inherently good at describing integrated visual regions, as they contain high-level semantic information.