ChatGPT has been trained using reinforcement learning from human feedback (RLHF). RLHF is similar to InstructGPT, a model presented at early 2022, but with some differences in the data collection process.