Advances in Very Deep Convolutional Neural Networks for LVCSR
Table 3: WER on the SWB part of the Hub5’00 testset.
['Version (Fig  1 )', 'SWB-1 (300h) CE', 'SWB-1 (300h) ST', 'SWB (2000h) CE', 'SWB (2000h) ST']
As expected, for architecture (b) with batch normalization we do not obtain good performance from sequence training, since we have to resort to spliced (inefficient) evaluation. The performance is worse with than without BN. In contrast, using the efficient convolutional evaluation with architecture (c), using batch normalization improves performance from 10.8 to 10.5 on SWB-1 (300h), matching the performance of the superior architecture (a). On SWB (2000h) adding BN brings the WER down to 9.5, almost matching the result of (a). We conclude that model (c) with BN recovers the lost performance from model (a).