Learning to Speak Fluently in a Foreign Language:Multilingual Speech Synthesis and Cross-Language Voice Cloning
Table 5: Effect of EN speaker cloning with no residual encoder.
['Model', 'Target Language EN', 'Target Language ES', 'Target Language CN']
Naturalness MOS decreases by 0.4 points for EN-to-CN cloning without the residual encoder (bottom row). In informal comparisons of the outputs of the two models we find that the model without the residual encoder tends to skip rare words or inserts unnatural pauses in the output speech. This indicates the VAE prior learns a mode which helps stabilize attention.