Multi-talker ASR for an unknown number of sources:Joint training of source counting, separation and ASR
Table 1: Source separation performance in SDRi in dB for varying numbers of talkers given the oracle number of sources. The mixtures contain full overlap only (min subset).
['Method', 'Method', 'number of talkers 1', 'number of talkers 2', 'number of talkers 3', 'number of talkers 4']
The OR-PIT is forced to the correct number of iterations. For the TasNet, only the K outputs with the largest energy are considered for evaluation. Fine-tuning the ASR system with the single-iteration scheme (12) can improve the performance, but the mismatch created by fine-tuning the FE this way degrades the performance compared to fine-tuning ASR (13). Using the multi-iteration fine-tuning scheme (14 and 15) can prevent this, but the overall performance is not comparable to the two-speaker TasNet model, although the OR-PIT achieves a better separation performance, according to Tab. This might be caused by the more complex training procedure.