Sequence-based Multi-lingual Low Resource Speech Recognition
Table 2: Word (% WER) and phoneme error rate (% PER) for each of the test languages, on the Babel conversational development test sets.
['Model', 'Kazakh WER', 'Kazakh PER', 'Turkish WER', 'Turkish PER', 'Haitian WER', 'Haitian PER', 'Mongolian WER', 'Mongolian PER']
To align with project goals, we chose to perform experiments on a set of four languages which are the closest/ have maximum phone overlap with Kurmanji – Kazakh, Turkish, Mongolian and Haitian. We used a 6-layer bidirectional LSTM network with 360 cells in each direction, which performed best on average across the majority of Babel languages in a systematic search experiment. For consistency, we used absolutely identical settings across all languages, and did not perform any language-specific tuning, other than choosing the lowest perplexity language model between 3-gram and 4-gram models for WFST-based decoding. Techniques such as blank scaling and applying a softmax temperature can often improve results significantly, but we did not apply any of them here for consistency. Again, this setup performed best on average on a larger set of languages. If we fine-tune the entire model towards each language specifically, performance improves further, by 4.4% on average over the baseline. If we roughly double the amount of training data by adding the Switchboard 300h training set to the “MLing” training data, performance improves yet again, for both the universal (MLing+SBWD) and language-specific (MLing+SWBD & FT) case. Overall, WER and PER improve by about 6% absolute (