Deep Speech 2: End-to-End Speech Recognition in English and Mandarin
Table 10: Comparison of English WER for Regular and Noisy development sets on increasing training dataset size. The architecture is a 9-layer model with 2 layers of 2D-invariant convolution and 7 recurrent layers with 68M parameters.
['Fraction of Data', 'Fraction of Data 1%', 'Hours 120', 'Regular Dev', 'Regular Dev 29.23', 'Noisy Dev', 'Noisy Dev 50.97']
Our English and Mandarin corpora are substantially larger than those commonly reported in speech recognition literature. This is done by randomly sampling the full dataset before training. For each dataset, the model was trained for up to 20 epochs though usually early-stopped based on the error on a held out development set. We note that the WER decreases with a power law for both the regular and noisy development sets. The WER decreases by ∼40% relative for each factor of 10 increase in training set size. We also observe a consistent gap in WER (∼60% relative) between the regular and noisy datasets, implying that more data benefits both cases equally.