Data Augmenting Contrastive Learning of Speech Representationsin the Time Domain
Table 4: Phone Error Rate (PER) in the semi-supervised setting. A linear classifier is added on top of Librispeech-100 pretrained CPC2 models and fine tuned with either 10min, 1h or 10h of Libri-light labelled data with a CTC loss. For comparison, reference Libri-light results plus the untrained CPC2 architecture fully fined-tuned with 10 h.
['System', 'Augmented fine-tuning', 'dev- clean', 'dev- other', 'test- clean', 'test- other']
For the supervised fine-tuning phase, we found out that we got the best results by using only pitch augmentation. Other methods having low or negative effects in this case. The combined effects of data augmentation on pretraining and fine-tuning adds up to 12-15% relative improvement across the different training sets. Interestingly, we find that with data augmentation we can beat the reference baseline (pretraining on 60k hours plus fine tuning on 10 hours) on frozen features with substantially less data (pretraining on 100 hours, plus fine tuning on 1 hour). Another point worth mentioning is that with data augmentation, 10 minutes of data on frozen features is sufficient to outperform the no-pretraining reference with 10 hours of labels.