MHSAN: Multi-Head Self-Attention Network for Visual Semantic Embedding
Table 3: Effect of self-attention (1-head) in each encoder. We denote the image encoder using selective spatial pooling and the text encoder using the last hidden state which are used in Engilberg [3] as W/O attention. We report only R@1 for effective comparison.
['Image Encoder', 'Text Encoder', 'Sentence Retrieval(R@1)', 'Image Retrieval(R@1)']
For this experiment, we use the MS-COCO dataset. To see the effect of the single-head self-attention mechanism at each encoder, we apply the single-head attention to each encoder separately. Visualization. We observe that self-attention with a single-head captures the missing parts on the selective spatial pooling, Through the visualization, we demonstrate that our model is more suitable for visual semantic embedding, where the encoding of rich details is necessary.