Curriculum Pre-training for End-to-End Speech Translation
Table 4: Ablation study on LibriSpeech expanded setting. ‘-’ indicates removing the task or phase from our method.
['Method', 'BLEU']
Ablation Study To better understand the contribution of each component, we perform an ablation study on LibriSpeech expanded setting. On the one hand, we show that both of our proposed pre-training tasks are beneficial: The performance drops when we remove either one of them. On the other hand, we show the two-phases pre-training paradigm is necessary: The “- phase 2” experiment degenerates to the simple ASR pre-training baseline. In “-phase 1” setting, we find that without the ASR pre-training, the training accuracy on FMLM task and FBLT task drops a lot, which further affects the ST performance. This means the ASR task is necessary for both the advanced courses and ST. In “Multi3” setting, we pre-train the model on ASR, FMLM and FBLT tasks in one phase. In this setting, we observe multi-task learning also decrease individual task performances (ASR, FMLM and FBLT) compared to curriculum learning. One reasonable expanation is that it is hard to train on the FMLM and FBLT tasks which takes masked input from randomly initialized parameters, which also leads to performance degradation on the ST task.