Adversarial Training for Multilingual Acoustic Modeling
Table 2: WERs (%) of multilingual models before and after fine-tuning.
['Language', 'WER (%) Multilingual', 'WER (%) +Fine Tuning']
We find that fine-tuning for multilingual models benefits nearly all languages. This means most languages benefit from further biasing the feature representations towards its own language while not losing shared knowledge from other languages. We only see degradation in English and it is because fine tuning results in loss of the shared knowledge due to its relatively large amount of data. We confirmed this by fine tuning only the AM layers for English and achieved a comparable WER (33.8%) as the multilingual model.