A COMPARISON OF END-TO-END MODELS FOR LONG-FORM SPEECH RECOGNITION
Table 1: Word-error-rates of end-to-end models on YouTube test set. For the Original, the utterances are segmented based on the appearance of silence. The resulting segmented utterances ranges from a few seconds to a few minutes long. Segment 16s corresponds to imposing an additional segmentation threshold on the silence-segmented utterances where utterances longer than 16 seconds are force segmented. We compare the results of using the overlapping inference with chunk size 8 and 16 seconds. *: the word-error-rates of CTC models are from [12] and are with language models.
['Model', 'Original', 'Segment 16 [ITALIC] s', 'Segment 30 [ITALIC] s', 'Overlapping inference 8 [ITALIC] s', 'Overlapping inference 16 [ITALIC] s']
As shown from the word-error-rates (WERs), attention models have problems scaling up to long utterances. In particular, those high WERs are due to the high deletion errors. For example, of the 67.1 WERs of the soft attention model, it consists of 63.2 deletion errors, 0.7 insertion errors, and 3.1 substitution error. This implies that on recognizing long-form utterances, attention models failed to generalize to the whole utterance and only produce transcripts for a sub-sequence of the utterance. The attention approaches that utilize monotonic alignment property all perform better than the vanilla soft-attention model, but they still exhibit serious problems generalizing to long utterances. Among all the attention mechanisms, the GMM-based monotonic attention model perform the best. The RNN-T model is robust to long-form utterances, and in fact achieves the best quality when longer utterances are being preserved. The attention models are significantly worse compared to the CTC models on long-form speech recognition, but the RNN-T model is able to outperform them.