Minimum Latency Training Strategiesfor Streaming Sequence-to-Sequence ASR
Table 4: Ablation study on the decoder side
['Model', 'WER [%]', 'Corpus-level latency [frame] (↓) Ave.', 'Corpus-level latency [frame] (↓) Mid.', 'Corpus-level latency [frame] (↓) 90th', 'Corpus-level latency [frame] (↓) 99th']
Quantity loss was not necessary for the baseline MoChA and MinLT, but essential for DeCoT. Warm start training was also necessary for both DeCoT and MinLT. This is probably because incorrect error signals were propagated into the model in the early training stage when training from scratch. The combination of DeCoT and MinLT with warm start training degraded WER too much although the latency was reduced further. The baseline WER was indeed boosted thanks to more updates by warm start training at the cost of latency, but DeCoT with δ=16,24 were still better than it with much smaller latency. We also tried to directly shift the boundary locations without boundary supervisions by setting bi to 0 for all i in Eq. However, this did not lead to the latency reduction, from which we can confirm the effectiveness of our proposed expected latency objective.