Semi-Supervised Training for Improving Data Efficiency in End-to-End Speech Synthesis
Table 1: MCD between ground-truth audio and synthesis from 7 Tacotron variants (lower is better). For T-Enc, we include both the results of using NNLM (1st row) and W2V (2nd row) as the word embedding module; concatenation/attention and input/top denote the conditioning method and location, respectively. The best result is marked in bold.
['T-Base', 'T-Enc concatenation', 'T-Enc concatenation', 'T-Enc attention', 'T-Enc attention', 'T-Dec', 'T-Enc-Dec']
We first compare the four configurations of encoder conditioning. Here we include the results of both NNLM and W2V word embeddings. We can see that models using NNLM always outperform their W2V counterpart. We speculate that this is because W2V only conveys word meanings but not the contextual or structural information, which is modeled in NNLM. Among them, T-Dec achieves the best result. However, T-Enc, T-Dec, and T-Enc-Dec achieve similar MCD results (12.46, 12.09, 12.27, respectively). As shown in side-by-side comparisons below, the raters did not strongly prefer one over the other two, either.