Minimum Latency Training Strategiesfor Streaming Sequence-to-Sequence ASR
Table 3: Results of latency reduction strategies on the decoder side
['Model', 'WER [%]', 'Corpus-level latency [frame] (↓) Ave.', 'Corpus-level latency [frame] (↓) Mid.', 'Corpus-level latency [frame] (↓) 90th', 'Corpus-level latency [frame] (↓) 99th']
5.2.3 Leveraging hard alignments in the decoder We then leverage the hard alignments on the decoder side. We initialized all models expect for the baseline with the baseline MoChA (warm start) since alignment constraints in the decoder makes training MoChA from scratch much harder. Note that we did not provide framewise supervision to the encoder in these experiments. Both DeCoT and MinLT significantly reduced the latency. An interesting observation was that WER also improved significantly at the same time with DeCoT with δ≥16. We obtained 8.0% and 10.6% relative WER improvements with δ=16 and 24, respectively. One possible explanation is that clean paths were emphasized more when out-of-boundary paths (potentially noisy) were removed during training. DeCoT has the effect to reduce the outlier as confirmed from the drastic latency reduction in the tail parts, but too much constraint with the small δ collapsed the model. In contrast, MinLT is effective for moving the center of latency distributions to the left side since the median improved by 40%. Note that when the i-th boundary corresponding to a non-EOS token is not activated until the last input timestep T (i.e., αi,j<0.5), we set the boundary to T to calculate the latency.