Exploring Phoneme-Level Speech Representations for End-to-End Speech Translation
Table 3: Comparing effects of target preprocessing with different sources on BLEU, Spanish-English 40hr
['Target Preproc.', 'Target Length', 'Frames dev', 'Frames test', 'Phonemes dev', 'Phonemes test']
Previous work on sequence-to-sequence speech translation has used encoder downsampling of 4×, while 8× is more common among sequence-to-sequence ASR systems Zhang et al. We use encoder downsampling of 4×, concatenating adjacent states after each layer. By averaging frames per local phoneme label in addition to encoder downsampling, source sequence lengths are further reduced on average by 79%, yielding final encoder state lengths of 22, closest in length to 1k BPE targets (14) rather than characters (50). Given that the 1k BPE model performs best, it does appear that more similar source and target lengths boost performance.