Robust Training of Vector QuantizedBottleneck Models
TABLE I: Higher codebook usage (perplexity) leads to lower phoneme error rate (PER) on the supervised WSJ task
['Model', 'Dev PER', 'Code Perplexity']
We notice that enabling the vanilla VQ bottleneck hurts the performance of the model. Batch normalization together with separate codebook learning rate help, but best results also require data-dependent codebook reestimation. This model not only offers the highest codebook usage (indicated by highest perplexity), but also reaches the error rate below the continuous baseline, demonstrating the regularizing effect of the information bottleneck.