Prosody Transfer in Neural Text to Speech Using Global Pitch and Loudness Features
Table 1: MOS Naturalness scores for the models and preference score for prosody transfer. Higher is better.
['Model', 'GT', 'TC2', 'GS-TC2']
4.2.1 Subjective Evaluations We begin with a MOS evaluation of speech naturalness. We use text from the test set and generate speech using the baseline TC2 and the GS-TC2 model. We also evaluate the naturalness of ground truth (GT) recorded test audio. Note that for the GS-TC2 model, we use the corresponding audio’s ground truth prosody features for conditioning. For each MOS test, a total of 20 listeners consisting of English speakers were selected. Each listener was presented with 9 audio files and was asked to rate the naturalness of these files on a scale of 1-5 with 0.5 increments. The addition of the prosody transfer module results in a small decrease in the naturalness of the TC2 model. This issue is not frequently observed in the TC2 model’s outputs because the TC2 model converges to the dominant prosody in the speech dataset, which is a neutral tone. Next, we evaluate prosody transfer from the reference speech to the synthesized speech for an arbitrary line of text. This is a side-by-side test comparing synthesized speech from two models which was first proposed by Skerry-Ryan et al. We ask listeners to rate which of the two synthesized samples is more similar to a reference speech sample. Specifically, we construct triplets (A,X,Y) from the test set. A is the reference speech sample, X and Y are utterances synthesized using the same text but using TC2 and GS-TC2 respectively. For this test, a total of 27 listeners consisting of English speakers were selected. Each listener was presented with 6 triplets and asked to rate which synthesized waveform matches more closely to the reference in terms of prosody, on a Likert scale of -3 to 3 with increments of 1. Here, -3 implies X is closer to A, +3 implies Y is closer to A, and 0 implies that there is no difference. It can be inferred that the GS-TC2 model successfully transfers prosody to synthesized audio compared to baseline TC2.