Reducing Bias in Production Speech Models
Table 1: Comparison of CTC and GramCTC.
['Loss', 'Train CER', 'Train WER', 'Train\xa0Holdout CER', 'Train\xa0Holdout WER', 'Dev CER', 'Dev WER']
GramCTC effectively reduces the learning burden of ASR network in two ways: 1) it decomposes sentences into pronunciation-meaningful n-grams, and 2) it effectively reduces the number of output time steps. Both aspects simplify the rules the network needs to learn, thus reducing the required network capacity of the ASR task. There are some interesting distinctions. First, the CERs of GramCTC are similar or even worse than CTC; however, the WERs of GramCTC are always significantly better than CTC. This is probably because GramCTC predicts in chunks of characters and the characters in the same chunk are dependent, thus more robust. Secondly, we also observe the performance on the dev set is relatively worse than that on the train holdout. Our dev dataset is not drawn from the same distribution of the training data - this exhibits the potential for GramCTC to overfit even a large dataset.