MHSAN: Multi-Head Self-Attention Network for Visual Semantic Embedding
Table 5: Impact of diversity regularization on performance of retrieval. The lower the diversity loss, the more diverse the local regions focused by the multiple attention maps.
['Diversity Regularization', 'Sentence Retrieval', 'Image Retrieval', 'Diversity Loss']
In this subsection, we experiment to observe the effect of diversity regularization in the loss. We conjecture that diversity regularization encourages encoded vectors to contain various semantic meanings in an image, reducing the redundancy of the encoded vectors and attention weight vectors. Visualization. We qualitatively present the multiple attention maps focusing on more distinct local regions by introducing the diversity regularization. Therefore, the image embedding encodes more distinct components in an image, resulting in an improvement of performance on the image-text retrieval tasks.