CSTNet: Contrastive Speech Translation Network for Self-Supervised Speech Representation Learning
Table 5: Phoneme Error Rate using multiple self-supervised learning methods.
['[BOLD] Method', '[BOLD] PER']
Representations learned by CSTNet significantly outperform all other systems except Wav2Vec, where the best CSTNet system trained on English-French lags by 8 percentage points (pp). Our worst model, English-German, outperforms ResDAVENet by 6pp. For ResDAVENet, we compute the PER using features from every layer and report the best results. This is an encouraging result showing that CSTNet can learn better linguistic information than the ResDAVENet trained on the audio-visual retrieval task. We do not train any of the comparison features extractors on our own, but use the publicly available checkpoints released by the authors of the respective methods.