The IBM 2015 English Conversational Telephone Speech Recognition System
Table 2: Comparison of word error rates for CE-trained DNNs with different number of outputs and phonetic context size on Hub5’00 SWB.
['Nb. outputs', 'Phonetic ctx.', 'WER SWB (CE)']
When training on 2000 hours of data, we found it beneficial to increase the number of context-dependent HMM output targets to values that are far larger than commonly reported. We conjecture that this is because GMMs are a distributed model and require more data for each state to reliably estimate the mixture components, whereas the DNN output layer is shared between states. This allows DNNs to have a much richer target space. Additionally, we experimented with growing acoustic decision trees where the phonetic context is increased to heptaphones (±3 phones within words and ±2 phones across words).