[
Table 3: Results of scheduled sampling methods on MSVD with different sets of semantic features. A larger mAP implies a better representation of semantic meanings.
['Semantic Features (mAP)', 'B-4', 'C', 'M', 'R', 'Overall']
Semantic features are the output of a multi-label classification task. Mean average precision (mAP) is often used to evaluate the results of multi-label classification task (tsoumakas2007multi-label). And we apply it to evaluate the quality of semantic features. We can clearly infer from them that a better multi-label classification result results in a better video captioning model. Semantic features with higher mAP provide clearer potential attributes of a video for the model. Thus, the model is able to generate better video annotations by considering semantic features, spatio-temporal features and context information comprehensively. Teacher Forcing utilizes human annotations to guide the generation of words during training and samples from the word distribution of the output of the model to direct the generation during inference. argmax gradually transfers from teacher forcing way to sample words with the largest possibility from the model itself during training. Multinomial is close to argmax but samples words at random from the distribution of the model at each step. Our method explores RNN state space in larger scope and thus, is likely to find a lower local minimum during training.