Direct speech-to-speech translation with a sequence-to-sequence model
Table 1: Dataset-specific model hyperparameters.
['[EMPTY]', 'Conversational', 'Fisher']
Given the small size of the dataset compared to that in Sec. The model was especially sensitive to the auxiliary decoder hyperparameters, with the best performance coming when passing activations from intermediate layers of the encoder stack as inputs to the auxiliary decoders, using slightly more aggressive dropout of 0.3, and decaying the auxiliary loss weight over the course of training in order to encourage the model training to fit the primary S2ST task.