What does a Car-ssette tape tell?
Table 4: Results of the baseline model trained on 3 datasets.
['[EMPTY]', 'BLEU4 Model', 'BLEU4 Human', 'BERT Model', 'BERT Human']
In order to verify our model’s generalization capabilities, we trained the baseline model on all 3 datasets. Firstly, our model is capable of being generalized to other datasets, in particular the cross-scene dataset: both the BLEU4 and the BERT similarity score on Joint Dataset are relatively good, meaning that the baseline model can distinguish different scenes. Due to the fact that every audio in hospital scene only had 3 annotations, our model showed its preference towards the car scene: it mistakenly generated hospital-related captions for only 12 car inputs and 308 car-related captions for hospital inputs. Secondly, the current model’s capability of generalizing more unique sentences was verified. Thirdly, both BLEU4 and the BERT similarity score showed that though scene-specific and non-repetitive sentences can now be generated, a great discrepancy between human and machine outputs still existed. More future research is needed to investigate the specific drawbacks and eventually make captioning tasks feasible.