Synchronous Speech Recognition and Speech-to-Text Translation with Interactive Decoding
Table 2: The performance of speech recognition and speech translation under different hyper-parameters λ on the En-Zh Dev set and Test set.
['[ITALIC] λ', 'Dev WER', 'Dev BLEU', 'Test WER', 'Test BLEU']
We investigate how much information from two tasks should be taken into consideration in the interactive attention sub-layer. If λ=0.0, the model degrades to traditional multi-task learning model which does not utilize any information from the other task. As shown in the table, as λ increases, both recognition quality and translation quality can be improved with information interacting. When λ=0.3, our interactive learning model achieves the best performance on the speech translation task. However, λ can not be too large, otherwise two tasks may interfere with each other and affect its own performance. Therefore, we use λ=0.3 for all experiments.