DiscreTalk: Text-to-Speech as a Machine Translation Problem
Table 1: Training conditions.
['[BOLD] VQ-VAE training condition Sampling rate', '[BOLD] VQ-VAE training condition 24,000 Hz']
To demonstrate the performance of the proposed model, we conducted an experimental evaluation using the JSUT corpus [sonobe2017jsut]. The JSUT corpus includes 10 hours of a single female Japanese speech. We used 7,196, 250, and 250 utterances for training, validation, and evaluation, respectively. The speech was downsampled to 24k Hz and the input text was converted into a phoneme sequence using Open JTalk [openjtalk]. To check the performance of the proposed method, we compared the following five models: Target: The target speech. We downsampled to 24k Hz and trimmed the silence at the beginning and the end of utterances by the force alignment with Julius [lee2009recent]. Baseline: The baseline system using Transformer-TTS [li2018transformer] with Parallel WaveGAN [Yamamoto2020]. We used the open-source toolkit ESPnet-TTS [hayashi2020espnet] to build this system. Reconst: Reconstructed speech by the proposed VQ-VAE. Proposed (Raw): The proposed model with raw discrete symbols as the target of the NMT model. Proposed (SW): The proposed model with subword units. For the proposed models, we built several models using different downsampling factor (DSF) in the VQ-VAE model (DSF128 or DSF256) and the different number of subword units (SW256 or SW512)