Improvements to Deep Convolutional Neural Networks for LVCSR
Table 10: WER of HF Sequence Training + Dropout
['Non-Linearity', 'WER']
By using dropout but fixing the dropout mask per utterance across all CG iterations, we can achieve a 0.6% improvement in WER. Finally, if we compare this to varying the dropout mask per CG training iteration, the WER increases. This shows experimental evidence that if the dropout mask is not fixed, we cannot guarantee that CG iterations produce conjugate search directions for the loss function.