There were low hanging fruit that allowed that rapid speedup (quadratic memory of basic transformers; poor pytorch default efficiency; and slow step convergence of original stable diffusion). Sped up via mem eff. attention, TensorRT, and distil