A Convolutional Deep Markov Modelfor Unsupervised Speech Representation Learning
Table 1: Phone Classification (FER) and Recognition (PER) when trained on a subset of Wall Street Journal. Suffixes -50, -360, -960 denote the amount of hours of LibriSpeech training data used for training the unsupervised model.
['% of Labeled Data', '1% FER', '1% PER', '2% FER', '2% PER', '5% FER', '5% PER', '10% FER', '10% PER', '50% FER', '50% PER', 'Low Shot (0.1%) PER']
recognition (PER) error rates on the WSJ eval92 dataset for different representation learning techniques. ConvDMM is trained on Mel-Frequency Cepstral Coefficients (MFCCs) with concatenated delta and delta-delta features. ConvDMM-50 and PASE-50 are both trained on 50 hours of LibriSpeech, ConvDMM-960, Wav2Vec-960 and MockingJay-960 are trained on 960 hours. ConvDMM-360 is trained on the 360 hours of the clean LibriSpeech dataset. RDVQ is trained on the Places400k spoken caption dataset [harwath2016unsupervised]. We do not train any of the representation learning systems that are compared against ConvDMM on our own. We use the publicly available pre-trained checkpoints to extract features. The linear classifiers used to evaluate the features extracted from unsupervised learning systems are trained on different subsets of WSJ train dataset, ranging from 4 mins (0.1%) to 40 hours (50%). To study the effect of modeling temporal structure in the latent space as in ConvDMM, we train a Gauss VAE which is similar to the ConvDMM except that it does not contain the transition model and hence, is a traditional VAE with isotropic Gaussian priors over the latent states [kingma2013auto].