TERA: Self-Supervised Learning of Transformer Encoder Representation for Speech
TABLE VII: Ablation Study. We use phone classification, speaker recognition and ASR results to study the effect of different auxiliary objectives, including comparison of pre-training with unidirectional or bidirectional context, channel objective that helps learn speaker identity (highlighted in grey), and pre-training with different acoustic features other than fMLLR (MFCC and FBANK). All of the models are pre-trained on the LibriSpeech train-clean-100 subset, and the numbers are testing results in percentage.
['[EMPTY]', 'Method', 'Bidirectional Context', 'Phone Classification Linear', 'Phone Classification 1 Hidden', 'Speaker Recognition Frame', 'Speaker Recognition Utterance', 'Speech Recognition WER', 'Speech Recognition Rescore']
We perform ablation study to better understand the performance of multi-target auxiliary objectives adopted in the TERA pre-training. For each downstream task, the output of the last layer of TERA is used as the extracted representation if not specified otherwise (no WS or FT). All the reported numbers are percentage and evaluated on test set. The TERA models investigated here are the base architectures pre-trained with 100 hours of LibriSpeech.