Towards Language-Universal End-to-End Speech Recognition
Table 3: CER obtained for a monolingual model with different pre-training strategies. The model was then fine-tuned on 150 hours of the target language (DE), except * where no fine-tuning was performed.
['Pre-training Training Data', 'Pre-training Hours', '% CER DE']
While the primary goal of this work was to create a language-universal model, we found that this model is also a good initial model for creating a language-specific monolingual model when the training data is limited. Notably, initializing with the gated language-universal model outperformed all of the other approaches, including bootstrapping from an EN model trained with significantly more data. The second best performance was obtained by the gated language-universal model directly, even without further fine-tuning on the DE training data.