Exploring Phoneme-Level Speech Representations for End-to-End Speech Translation
Table 2: Comparison of frame vs phoneme input on Spanish-English SLT, with average BLEU improvement and average reduction in training time.
['[BOLD] Data', 'Frames  [BOLD] dev', 'Frames  [BOLD] test', 'Phonemes  [BOLD] dev', 'Phonemes  [BOLD] test', 'BLEU Δ', 'Time Δ']
On our target task, we compare different subsets of the data to see how our method compares under different data conditions, using the full 160 hours as well as 40 and 20 hour subsets. When we use our phoneme-like embeddings, we see relative performance improvements of 13% on all data sizes, or up to 5.2 BLEU on the full dataset. Further, in reducing source lengths by ∼80%, training time is improved.