Towards unsupervised speech recognition and synthesiswith quantized speech representation learning
Table 1: Phoneme error rate (%) on different amount of paired data.
['Method', '20 min', '15 min', '10 min', '5 min']
To perform speech recognition, we selected the most possible phoneme sequence according to the distance between each encoder output and phoneme embedding in the codebook (see Eq. with beam search and trimmed the repeated phonemes. The baseline is an ASR model (which is not required to reconstruct the input speech nor to learn any representation) having the same architecture as our encoder with an additional projection layer to predict probability over phoneme set.