Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesis
Table 6: Speech from fictitious speakers compared to their nearest neighbors in the train sets. Synthesizer was trained on LS Clean. Speaker Encoder was trained on LS-Other + VC + VC2.
['Nearest neighbors in', 'Cosine similarity', 'SV-EER', 'Naturalness MOS']
Bypassing the speaker encoder network and conditioning the synthesizer on random points in the speaker embedding space results in speech from fictitious speakers which are not present in the train or test sets of either the synthesizer or the speaker encoder. Even though these speakers are totally fictitious, the synthesizer and the vocoder are able to generate audio as natural as for seen or unseen real speakers. The low cosine similarity to the nearest neighbor training utterances and very high EER indicate that they are indeed distinct from the training speakers.