Learning to Caption Images with Two-Stream Attention and Sentence Auto-Encoder
Table 1: Evaluation scores for our image captioning model in Figure 3 (a) with the soft-attention module and, ablations of our two-stream attention module.
['Models', 'Categories', 'B-1', 'B-2', 'B-3', 'B-4', 'METEOR', 'ROUGE-L', 'CIDEr']
We replace this attention mechanism with our proposed one and evaluate its performance for different number of latent categories (C). The models with latent categories 64, 128 and 256 outperform the baseline, setting C to either of 128 and 256 result in significant improvements. The higher CIDEr score for 128 shows the model’s capability to generate more descriptive human-like sentences. We also observe that the model with 256 categories successfully localizes more subtle semantics such as “holding”.