Abstract
Table 4: Pearson correlation (r) of scores for systems produced in two separate data collection runs on Mechanical Turk.
['[EMPTY]', 'raw', '[ITALIC] z']
In order to investigate the reliability of the results produced by the new DA video captioning evaluation, we re-ran the evaluation in an entirely separate and repeated data collection/assessment on Mechanical Turk and compared results. The very high correlation between scores for systems confirms that the method of evaluation is reliable, producing system rankings that are almost perfectly reproducible for evaluation of video captioning.