CTC-synchronous Training for Monotonic Attention Model
Table 4: Librispeech results. ♣Quantity regularization is used. ♢SpecAugment is used.
['Model', 'Model', '%WER clean', '%WER other']
When using the unidirectional LSTM encoder for MoChA, we obtained 11.3% and 6.2% relative improvements with CTC-ST on the test-clean and test-other sets, respectively. As the training data size is much larger and the domain is not spontaneous, the gains from quantity regularization for the BLSTM encoder and CTC-ST for the LC-BLSTM encoder were smaller than TEDLIUM2, although both were still beneficial. CTC-ST was effective for obtaining gains from SpecAugment and led to additional improvements of 7.6% and 17.8% on the test-clean and test-other sets, respectively. The model parameter size was 53.77M and fixed through all experiments. Our optimal model requires only 830ms (= 400ms (Nc) + 400ms (Nr) + 30ms (CNN)) latency and the decoding complexity is linear.