End-to-End Multi-Speaker Speech Recognition using Speaker Embeddings and Transfer Learning
Table 3: Results (WER, %) of the baseline ASR, speaker embeddings conditioning and transfer learning on the two- and three-speaker overlapped speech datasets.
['[EMPTY]', 'wsj0-2mix', 'wsj0-2mix', 'wsj0-3mix', 'wsj0-3mix']
We evaluate two transfer learning approaches, namely parameters transfer and multi-condition training, for improving ASR performance on overlapped speech by utilization of non-parallel clean speech training data. The most striking observation to emerge from the results is that the training process has not converged on the dataset with three overlapping speakers, but the system has been able to decode the same recordings when trained on the combination of overlapped and clean speech datasets. This finding can be attributed to higher complexity of the modeled function in case of increased number of overlapping speakers, which the neural network could not learn just from the overlapped speech data, and demonstrates how crucial the role of the transfer learning approach for the solution of certain problems can be. The transfer learning results are expectedly better on the dataset with two overlapping speakers as well, especially for the open speaker condition, what is due to the number of additional speaker embeddings in the training data and subsequent better generalization of the relationship between speaker embedding and relevant acoustic features. Finally, multi-condition training has demonstrated slightly better results than parameters transfer. This finding is in agreement with the previous reports on transfer learning applications in ASR.