Learning Hierarchical Discrete Linguistic Units from Visually-Grounded Speech
Table 2: ABX scores and RLE bitrates for various SNRs on the noisy ZeroSpeech19 English test set. “R-B” stands for “RLE-Bitrate,” and (n) denotes a model trained on the noisy Places Audio dataset. For the WaveNet-VQ models, (ZS) and (PA) respectively denote training on the ZeroSpeech 19 English training set, and the clean Places Audio dataset.
['Model', 'Layer', 'Clean ABX', 'Clean R-B', '20-30 dB ABX', '20-30 dB R-B', '10-20 dB ABX', '10-20 dB R-B', '0-10 dB ABX', '0-10 dB R-B']
Training and testing on noisy data In Hsu et al. Here, we examine whether this robustness is also exemplified by the quantized versions of this model. For each ZeroSpeech testing waveform, we randomly sampled an AudioSet waveform of the same duration and performed linear mixing with a signal-to-noise ratio (SNR) selected randomly within a specified range. We construct low, medium, and high noise testing sets, corresponding to SNRs of 20-30 dB, 10-20 dB, and 0-10 dB. We find that for all models, a worsening SNR results in a deterioration in ABX performance. However, the ResDAVEnet-VQ models prove to be far more noise robust than the Wavenet-VQ model; even in the high noise testing set, the best ResDAVEnet-VQ model achieves an ABX of 23.96%, while the WaveNet-VQ models degrade to nearly-random ABX scores of 42.55% and 42.67%. Given that a ResDAVEnet-VQ model trained on the “clean” Places Audio captions is highly robust to additive noise on the ABX discrimination task, we investigated whether adding noise to the Places Audio captions themselves would result in an even higher degree of noise robustness. ZeroSpeech ABX tasks. In general, we observe a degradation ABX score in the clean conditions, but with a significantly higher degree of noise robustness in the noisier conditions.