End-to-End Multi-Speaker Speech Recognition using Speaker Embeddings and Transfer Learning
Table 4: Results (WER, %) of the proposed method depending on the training data.
['Training data', 'wsj0-2mix', 'wsj0-2mix', 'wsj0-3mix', 'wsj0-3mix']
As our method does not utilize any explicit knowledge about the number of overlapping speakers, the models should theoretically also work for testing data with larger or smaller number of overlapping speakers than in the training data. We test whether this is true in practice by decoding wsj0-3mix testing data with the model trained on wsj0-2mix training data (combined with LibriSpeech 100) and vice versa. Encouraged by the success of the previous transfer learning experiments, we also train a system on a combination of clean and overlapped two- and three-speaker datasets. In general it seems that the proposed method does not depend on the number of overlapping speakers but can benefit from training on larger amount of different conditions of speech overlap. A possible explanation of the slightly worse result of the best system on the open speaker set condition with two overlapping speakers might be a bias towards WSJ0 speakers in the combined training dataset introduced by the addition of wsj0-3mix dataset, which is slightly larger than wsj0-2mix dataset and therefore has more impact on two-speaker testing data than wsj0-2mix on three-speaker testing data in this experiment.