SpecAugment: A Simple Data Augmentation Methodfor Automatic Speech Recognition
Table 4: Switchboard 300h WER (%) evaluated for LAS-4-1024 trained with schedule B with varying augmentation and Label Smoothing (LS) policies. No LMs have been used.
['[BOLD] Policy', '[BOLD] LS', '[BOLD] SWBD', '[BOLD] CH']
We train LAS-4-1024 with policies (None, SM, SS) and schedule B. As before, we set the peak learning rate to 0.001 and total batch size to 512, and train using 32 Google Cloud TPU chips. Here the experiments are run with and without label smoothing. Not having a canonical development set, we choose to evaluate the checkpoint at the end point of the training schedule, which we choose to be 100k steps for schedule B. We note that the training curve relaxes after the decay schedule is completed (step sf), and the performance of the network does not vary much. We see that label smoothing and augmentation have an additive effect for this corpus.