Surprisingly, the architecture of ChatGPT is pretty simple. It's leveraging dumb computed, stacked high. It's betting that fast compute with a simple algorithm that leverages itself will be better than anything humans can come up with heuristically.