Unsupervised acoustic unit discovery for speech synthesis using discrete latent-variable neural networks
Table 2: Human and machine ZRSC’19 evaluation on the English and Indonesian test data. Higher MOS and similarity scores are better, while lower is better for the other metrics.
['Model', 'CER (%)', 'MOS [1, 5]', 'Similarity [1, 5]', 'ABX (%) latent', 'ABX (%) output', 'Bitrate']
Based on the development experiments on English, we submitted two versions of the VQ-VAE model for official evaluation to the ZRSC’19: the one uses ×4 and the other ×8 downsampling. We did this to measure the impact of bitrate on speech synthesis quality. Our approach performs worse on MOS and similarity compared to DPGMM-Merlin. On CER, however, we perform similarly or better in all cases. In terms of ABX directly on the discrete latent features, both VQ-VAE-x8 and VQ-VAE-x4 outperform the baseline—on English the latter even outperforms the supervised topline system (27.6% compared to 29.9% in ABX). On English, output features from both our models outperform the supervised topline system while on Indonesian the VQ-VAE-x4 model does. We use FFTNet mainly for computational reasons. (∼3.2) might indicate that we are reaching a performance ceiling with this vocoder.