Contents
Table 6.11: Comparison between existing image text matching systems. R@K is Recall@K (high is good). In our models, for image annotation we report R@K: any_txt and for image search we report R@K: avg_txt.
['[BOLD] Flickr30K: Comparison Models', '[BOLD] Flickr30K: Comparison image annotation', '[BOLD] Flickr30K: Comparison image annotation', '[BOLD] Flickr30K: Comparison image annotation', '[BOLD] Flickr30K: Comparison image search', '[BOLD] Flickr30K: Comparison image search', '[BOLD] Flickr30K: Comparison image search']
When comparing the results it is interesting to note that most recent research has gone into complicated modeling on the textual domain. They also try other types of recurrent and recursive neural networks. In contrast we use simple textual model and achieve comparable results. The bulk of our model’s strength comes from the visual model and the training methodology which specializes the performance towards a particular task boosting its performance. In our experiments we notice that temporal dependency (or information about word order) modeled with simple bi-grams or tri-grams is enough to achieve good results. The simplest addition to our model would be to turn on updating for our CNN while we are training on the image-text data set. This is called finetuning, since the CNN is pretrained on image data alone. However this will come at a cost of significantly additional training time, hence it is a trade off.