phi-LSTM: A Phrase-based Hierarchical LSTM Model for Image Captioning
Table 2: Vocab size, word occurrence and average caption length in train data, test data, and generated description on Flickr8k dataset.
['Number of sentence', '[BOLD] Train Data 30000', '[BOLD] Train Data 30000', '[BOLD] Test Data 5000', '[BOLD] Test Data 5000', '[BOLD] Test Data 1000', '[BOLD] Test Data 1000', '[BOLD] Gen. Caption 1000', '[BOLD] Gen. Caption 1000']
We are interested to know how well the corpus is trained by both models. Using the Flickr8k dataset, we computed the corpus information of i) the training data, ii) the reference sentences in the test data and iii) We remove words that occur less than 5 times in the training data, and it results in 4833 words being removed. However, this reduction in term of word count is only 2.48%. Furthermore, even though the model is evaluated in comparison to all reference sentences in the test data, there are actually 1228 words within the references that are not in our training corpus. Thus, it is impossible for the model to predict those words, and this is a limitation on scoring with references in all language models. For a better comparison with the 1000 generated captions, we also compute another reference corpus based on the first sentence of each test image. These numbers show that the actual number of words learned by these two models are barely 10%, suggesting more research is necessary to improve the learning efficiency in this field. Nevertheless, it shows that introducing the phrase-based structure in sequential model still improves the diversity of caption generated.