Towards Language-Universal End-to-End Speech Recognition
Table 1: Initial CER results for conventional single-task (stl), multi-task (mtl), and universal character set (univ) networks using monolingual and multilingual training data.
['Training', 'Total', 'Model', 'Test', 'CER', '% Rel.']
We first evaluated both multi-task and language-universal architectures on German and Spanish data, along with conventional monolingual training. In addition, we observed that models trained with multi-task learning with language-specific output labels and models trained with a universal label set performed comparably, demonstrating the potential of a universal label set. However, with a conventional BLSTM topology, the benefit gained from the multilingual data was far less than simply doubling the amount of training data from the target language to 300 hours. More concerning, the performance actually degraded when we increased the multilingual data from two to three language. These initial results indicate that simply training conventional models with multilingual data is not very helpful. However, we note that these results may differ in extremely low resource scenarios where the data available per language could be an order of magnitude smaller.