ChatGPT was created by first fine-tuning GPT3.5 on a dataset done by human labellers containing many prompts and answers. The bot is then further improved by continuously assigning rewards based on the quality of the output.