Learning to Speak Fluently in a Foreign Language:Multilingual Speech Synthesis and Cross-Language Voice Cloning
Table 2: Naturalness MOS of monolingual and multilingual models synthesizing speech of in different languages.
['Model', 'Input', 'Language EN', 'Language ES', 'Language CN']
For Mandarin, the phoneme-based model performs significantly better than char- or byte-based variants due to rare and OOV words. Compared to the monolingual system, multilingual phoneme-based systems have similar performance on ES and CN but are slightly worse on EN. CN has a larger gap to ground truth (top) due to unseen word segmentation (for simplicity, we didn’t add word boundary during training). The multispeaker model (bottom) performs about the same as the single speaker per-language variant (middle). Overall, when using phoneme inputs all the languages obtain MOS scores above 4.0.