Serialized Output Training for End-to-End Overlapped Speech Recognition
Table 4: WER (%) comparison of PIT and SOT. Test data were generated by mixing “test_clean”. Evaluation results with unmatched training/testing conditions were shown with parenthesis. Note that we trained the PIT model with up to 2 speakers since 3-output PIT required impractical training time.
['Model (1024-dim)', '# of Speakers in Training Data', '# of Params.', '# of Speakers in Test Data 1', '# of Speakers in Test Data 2', '# of Speakers in Test Data 3']
Note that we applied neither an additional language model nor feature-domain data augmentation In this experiment, we used 1024-dim model for both PIT and SOT. As shown in the table, PIT showed severe WER degradation for the 1-speaker case even when the training data contained many 1-speaker examples. SOT achieved significantly better results than the PIT model trained on 1- and 2-speaker mixtures for both 1- and 2-speaker evaluation data while recognizing 3-speaker evaluation data, with fewer parameters. For reference, the training speed for the 3-speaker SOT was roughly 30% faster (including I/O) than the 2-output PIT. The difference of the training speed would become much larger if we use PIT with more output branches.