My above assumption of how ChatGPT may have been implemented is based on the information I got from their blog and reading through InstructGPT's paper. The fundamental reason why GPT-3 is this great is because it includes humans in its training loop.