Harnessing Indirect Training Data for End-to-End Automatic Speech Translation: Tricks of the Trade
Table 6: Effect of fine-tuning (FT), both when added to the baseline or the pretrained (preT) baseline.
['Dataset AST + TTS', 'Bérard 12.58922', 'Δ FT 1.75475', '+ preT 15.19753', 'Δ FT 1.15312']
We apply fine-tuning whenever TTS-augmentation is used. Fine-tuning seems to mitigate the effect of domain shift introduced by the additional out-of-domain TTS data: in the Librispeech AST + TTS setup, fine-tuning improves by +1.7 (+1.2 BLEU on top of pretraining). For the MuST-C + MT setup, we see +2.3 BLEU ( +1.4 on top of pretraining).