Multilingual Speech Recognition with a Single End-to-End Model
Table 2: WER(%) of language-specific, joint, and joint+MTL LAS models.
['Language', 'Language-specific', 'Joint', 'Joint + MTL']
We first compare the language-specific LAS models with the joint LAS model trained on all languages. In fact, the joint model decreases weighted average WERs across all the 9 languages, weighted by number of words, by more than 21% relative to the monolingual models. This result is quite interesting not only because the joint model is a single model that is being compared to 9 different monolingual models, but unlike the monolingual models the joint model it not language-aware at runtime. Finally, the large performance gain of the joint model is also attributable to the fact that the Indian languages are very similar in the phonetic space Second, we compare the joint LAS model with the multitask trained variant. This might be due to the following reasons: (a) Static choice of λ. Since the language ID prediction task is easier than ASR, a dynamic λ which is high initially and decays over time might be better suited, and (b) The language ID prediction mechanism of averaging over encoder outputs might not be ideal. A learned weighting of the encoder outputs, similar to the attention module, might be better suited for the task.