DiscreTalk: Text-to-Speech as a Machine Translation Problem
Table 2: Character error rates calculated by the ASR model and token error rates of the NMT models.
['Method', 'CER [%]', 'TER [%]']
We conducted the objective evaluation using the ASR-based objective measure character error rate (CER) and the token error rate (TER) of the NMT models. As the ASR model, we used the Transformer-ASR model trained on the corpus of spontaneous Japanese (CSJ) [maekawa2003corpus]. A comparison between the different DSFs shows that the reconstruction with the small DSF achieved the performance comparable to the baseline while the use of the large DSF greatly affected the intelligibility. From audio samples available in [audio_samples], the model with a large DSF generated slurred speech while keeping high signal-to-noise (SN) ratio. The authors highly recommend listening to the samples to understand the difference. Thus, there was a trade-off between the size of DSF and speech articulation. From a comparison between the target types, the use of the subword was effective, especially in the case of the small DSF. This is because the same discrete symbol repeatedly appeared in the sequence due to high time resolution, and the subword can summarize these successive symbols. Note that since the use of the subword made the length of the target sequence smaller, it can also reduce the training time. However, the use of a large number of subword units made the target symbols sparse, decreasing the performance. Therefore, we need to tune the number of subword units according to the size of the training data. Focusing on the effectiveness of beam-search, it led to a slight improvement of the performance, but a large beam-size did not always bring the improvement. Interestingly, the predicted sequence is totally different from the ground-truth, and we could not find the meaningful correlation between the CER of the ASR model and the TER of the NMT model. The training process also tended to be over-fitting in the early stage. One of the possible reasons is that speech included various speaking styles or intonations, even for the same words. Therefore, the conversion from the text to the sequence of discrete symbols became a one-to-many problem.