Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesis
Table 7: Speech naturalness and speaker similarity Mean Opinion Score (MOS) with 95% confidence intervals of baseline models where the speaker encoder and synthesizer networks are trained jointly (top two rows). Included for comparison are the separately trained baseline from Table 5 (middle row) as well as the embedding lookup table baseline and proposed model from Tables 1 and 2 (bottom two rows). All but the bottom row, are trained entirely on LibriSpeech. The bottom row uses a speaker encoder trained on a separate speaker corpus. All evaluations are on LibriSpeech.
['System', 'Embedding Dim', 'Naturalness MOS Seen', 'Naturalness MOS Unseen', 'Similarity MOS Seen', 'Similarity MOS Unseen']
We find that both jointly trained models obtain similar naturalness MOS on Seen speakers, with the variant incorporating a discriminative speaker loss performing better on Unseen speakers. Finally, we note that the proposed model, which uses a speaker encoder trained separately on a corpus of 18K speakers, significantly outperforms all baselines, once again highlighting the effectiveness of transfer learning for this task.