Data Augmenting Contrastive Learning of Speech Representationsin the Time Domain
Table S2: Architecture ablations, ABX errors (Libri-light dev set). We compare modifications of the CPC architecture across different dataset sizes. In all cases, we apply the best data augmentation reported in the main text.
['[EMPTY]', 'Within spk. dev', 'Within spk. dev', 'Across spk. dev', 'Across spk. dev']
In this experiment, we test whether our data augmentation technique can build better speech features that can be used for downstream tasks. Here, we use the Libri-light limited supervision phone classification task [kahn2020], which contains intentionally small training sets (10 min, 1h or 10 hours of labelled data). We fine-tune a linear phone classifier built on top of the CPC features with a CTC loss (frozen features). On 10 hours of data, we also fine-tune the entire network. Again, we additionally experiment with an architecture that has a 3-layer LSTM (CPC2-L3) In the next experiment, we study the performance of our model in function of the size of the available data and the architecture size (controlled by the number of LSTM layers). We simulate the amounts of data available at ZeroSpeech2017 for Mandarin (3h), French (45h), and English (100h) by sub-sampling from LibriLight (3h and 45h) and using LibriSpeech (100h). In all experiments, we use the best data augmentation found in the main text (pitch+add+reverb-past).