Leveraging translations for speech transcription in low-resource settings
Table 2: Evaluating the output with word-level BLEU, multi-source models significantly improve upon the baselines on higher-resource settings (Spanish). On Ainu, the best model performs on par with the very competitive baseline.
['Source', 'Transcription BLEU Ainu', 'Transcription BLEU Spanish']
A higher BLEU almost always translates to lower WER. We only report results on Ainu and Spanish as Mboshi does not have standardized word segmentation rules. The BLEU scores reinforce our previous analysis. In the Ainu dataset, where a translation-only model is impossible to train (as outlined by the BLEU score of about 5.9), the multi-source model with shared attention mechanism performs on par with the speech-only model, as does the coupled ensemble model. In the CALLHOME dataset, our multi-source model with the tied attention mechanism achieves significantly higher BLEU scores than the translation-only or the coupled ensemble models. In addition, our best model’s WER on the CALLHOME test set is 53.0, outperforming LatticeTM that achieved a WER of 56.2, despite the fact that our model operates in a harder setting, trained and tested directly on speech.