Reducing Bias in Production Speech Models
Table 2: Performances and training efficiency of GramCTC with different model strides
['Loss Stride', 'WER 2', 'WER 4', 'Epoch Time (hours) 2', 'Epoch Time (hours) 4']
By striding over the input at a faster rate in the early layers, we effectively reduce the time steps of later layers, and reduce the training time in half. From stride 2 to stride 4, the performance also improves a lot probably because larger n-grams align with larger segments of utterance, and thus need lower time resolution.