Hal: Improved Text-Image Matching by Mitigating Visual Semantic Hubs
Table 1: Quantitative results on Flickr30k [51]. “ours” means our own implementation.
['#', '[BOLD] architecture', '[BOLD] loss', '[BOLD] image→text R@1', '[BOLD] image→text R@5', '[BOLD] image→text R@10', '[BOLD] image→text Med r', '[BOLD] image→text Mean r', '[BOLD] text→image R@1', '[BOLD] text→image R@5', '[BOLD] text→image R@10', '[BOLD] text→image Med r', '[BOLD] text→image Mean r', 'rsum']
Comparing Hal, Sum and Max. On Flickr30k, we experiment three models and Hal achieves significantly better performance than Max and Sum on the first two configurations. Interestingly, while Max fails badly on Flickr30k, it becomes very competitive on MS-COCO. This serves as an evidence of Max easily overfitting to small datasets. In conclusion, Hal maintains its edge over Max and Sum across regardless of data and architecture configurations. Even without global weighting (memory bank), Hal still beats the two triplet losses by a large margin. The equipment of memory bank can usually further boosts rsum by another 3−5. Also, it is worth noticing that Hal converges significantly faster than Max and Sum. Hal stabilizes after approximately 5 epochs while Max and Sum take roughly 10 epochs.