ChatGPT evolved from InstructGPT, which by itself is a successor of GPT3. OpenAI doesn't publicly state what was used to train these, but there have been some educated guesses that it has been mainly trained archives of the Web, corpora containin