Accelerating Hessian-Free Optimization for Deep Neural Networks by Implicit Preconditioning and Sampling
Table 2: Overall Training Time Improvements, Broadcast News
['Method', 'WER', 'Total Training Time (hrs)']
In this section, we combine the preconditioning and sampling to calculate the overall speedup in training time for BN. Overall we can see that PC+Gradient+CG sampling offers the fastest training time compared to the baseline. Training time is reduced from 68.7 hours to 44.5 hours, roughly a 1.5x speedup, with no loss in accuracy.