Multilingual Graphemic Hybrid ASR with Massive Data Augmentation
Table 2: WER results on each video dataset. Frequency masking is denoted by fm, speed perturbation by sp, and additive noise (Section 3.2) by noise. 3lang, 4lang and 7lang denote the multilingual ASR models trained on 3, 4 and 7 languages, respectively, as in Section 4.1. Lang-specific decoding denotes using multilingual acoustic model with language-specific lexicon and language model, as in Section 2.3. Average is unweighted average WER across 4 video types. Gain (%) is the relative reduction in the Average WER over each monolingual baseline.
['Language', 'Model', 'clean', 'noisy', 'xtrm1', 'xtrm2', 'Average', '% Gain']
In addition, combining both frequency masking and speed perturbation could provide further improvements. Secondly, we applied the 3 data augmentation techniques to the multilingual ASR of 7lang, and tested their additive effects. Note that on Kannada, we found around 7% OOV token rate on clean but around 10-11% on other 3 test sets, and we observed more gains on clean ; presumably because the improved acoustic model could only correct the in-vocabulary word errors, lower OOV rates therefore left more room for improvements. Hindi had around 2.5% OOV rates on each test set, and we found incorporating data augmentation into multilingual ASR led to on average 9.0% relative WER reductions.