ESPnet-TTS: Unified, Reproducible, and Integratable Open Source End-to-End Text-to-Speech Toolkit
Table 3: ASR-based CER results.
['Method', 'Sub [%]', 'Del [%]', 'Ins [%]', 'CER [%]']
First, we evaluated the E2E-TTS models using the ASR-based objective measure character error rate (CER). As the ASR model, we used the Transformer trained on the Librispeech dataset [panayotov2015librispeech]. A comparison between Tacotron 2 and Transformer TTS shows that the Transformer TTS caused more deletion errors than Tacotron 2. One of the possible reasons is that the multi-head attention is based on the dot product attention and therefore, the restriction of causality or continuity in the attention is weaker than the location-sensitive attention and the forward attention. Therefore, it is expected that the multi-head attention based on the other types of attention mechanisms will reduce the errors. FastSpeech.v2 achieved the best result among models, reducing all of the errors in comparison to its teacher model Transformer.v1. This is reasonable because FastSpeech is a non-autoregressive model which, theoretically, does not cause the repetition of the word [ren2019fastspeech]. However, FastSpeech.v3 is worse than FastSpeech.v2, especially in terms of the deletion errors. This degradation is caused by the error of the text processing front-end which converts characters into phonemes.