Abstract
Table 3: Direct Assessment human evaluation results of systems originally participating in the TRECVid 2016 VTT task; raw scores are average scores for systems; z scores are average standardized scores for systems; Human-b comprises one of the set of video captions produced by human annotators.
['System', 'raw (%)', '[ITALIC] z', '[ITALIC] n']
z score distributions for systems according to Wilcoxon rank-sum test. As expected, the human captions, Human-b, achieve the highest score overall, at 88.2%, and including such a human system allows an estimation of the score that could be expected from a system that has effectively solved the video captioning problem. Scores for the actual systems range from 37.7 to 66.2 %, showing the top system to be still some distance from human annotation quality. In terms of statistical significance, we see an almost absolute ranking of systems, with only a single pair of systems (INF & DCU) tied with no significant difference in their scores.