A Case Study on Combining ASR and Visual Featuresfor Generating Instructional Video Captions
Table 2: Caption generation performance: AT+Video is a multimodal model that adds visual frame features to AT. A bolded value in a column indicates a statistically-significant improvement, whereas an underline indicates a statistical tie for best (p
['[EMPTY]', 'BLEU-4', 'METEOR', 'ROUGE-L', 'CIDEr']
Generation Experiment Results. For unimodal models, simple baselines like FASC (filtered ASR) and RET (training-caption retrieval) outperform the state-of-the-art video-only model of \newcitesun2019contrastive, according to the four automatic evaluation metrics. Overall, AT yields the best unimodal performance. Combining ASR and visual signals into a multimodal representation performs even better: the AT+Video model tends to outperform AT (and \newcitesun2019contrastive), according to ROUGE-L, CIDEr, and METEOR (p 