My current obsession: How does one best document the architecture of a LLM such as GPT-3 or ChatGPT (where by "architecture" I mean "significant design decisions)? For example, GPT-3 has 96 attention layers, alternating between dense and sparse. Stay tuned...