The Long-Short Story of Movie Description
Table 5: Comparison of prior work and our proposed method using all popular evaluation measures. Human scores in form of ranking from 1 to 3, where lower is better. All results reported on the test Set of MPII-MD.
['Approach', 'Automatic Score  [ITALIC] CIDEr', 'Automatic Score  [ITALIC] BLEU@4', 'Automatic Score  [ITALIC] ROUGEL', 'Automatic Score  [ITALIC] METEOR', 'Human evaluation: rank Correct.', 'Human evaluation: rank Grammar', 'Human evaluation: rank Relev.']
Exploring different strategies to label selection and classifier training, as well as various LSTM configurations allows to obtain best result to date on the MPII-MD dataset. Human evaluation mainly agrees with the automatic measures. We outperform both prior works in terms of Correctness and Relevance, however we lose to S2VT in terms of Grammar. This is due to the fact that S2VT produces overall shorter (7.4 versus 8.7 words per sentence) and simpler sentences, while our system generates longer sentences and therefore has higher chances to make mistakes. For every test sentence we retrieve the closest training sentence according to the METEOR. The rather low METEOR score of 19.43 reflects the difficulty of the dataset.