Vector-quantized neural networks for acoustic unit discovery in the ZeroSpeech 2020 challenge
Table 2: Speaker classification results at probe points before and after quantization (shown under the “pre-quant” and “code” columns respectively).
['Model', 'Spkr. class. accuracy (%) code', 'Spkr. class. accuracy (%) pre-quant', 'ABX (%) code', 'ABX (%) aux']
To show that the VQ bottleneck discards speaker information, we analyze representations before and after quantization. At each probe point, we train a multilayer perceptron with 2048 hidden units to predict the speaker identity. We use mean-pooling after the non-linearity to aggregate features. Based on the drop in speaker classification accuracy across the probe points, the VQ layer clearly acts as an information bottleneck, forcing the models to discard speaker details. Interestingly, CPC without vector quantization performs well on ABX tests but does not explicitly discard speaker information. As a result, CPC alone was not capable of voice conversion in our experiments. Within-speaker sampling results in better speaker invariance (lower speaker classification accuracy) and significantly lower ABX scores (13.4% vs. 36.2%).