Accelerating Hessian-Free Optimization for Deep Neural Networks by Implicit Preconditioning and Sampling
Table 1: Total CG runtime for different quasi-Newton PC schemes
['Method', 'Loss', 'HF Iterations', 'Time (min)']
The Figure indicates that that all preconditioning methods require less time for CG, particularly as the number of total HF iterations increases (and thus the number of CG iterations increases).