Synchronous Speech Recognition and Speech-to-Text Translation with Interactive Decoding
Table 3: The performance of speech recognition and speech translation with different word latency in wait-k policy on the En-Zh Dev set and Test set.
['Wait- [ITALIC] k', 'Dev WER', 'Dev BLEU', 'Test WER', 'Test BLEU']
We then investigate the effect of word latency in wait-k policy on En-Zh language pairs. It indicates that the speech translation task can become easier if more source information from the same modality is given. However, as k increases, it will affect the performance of speech recognition task. If k→∞, this model degrades to the analogous two-stage model. Then the speech translation task can obtain the information from complete transcribed sentence, while speech recognition task can not utilize any information from translations. The interactive learning model has the best performance when k=3.