OpenAI, in a blog post, explained how it made ChatGPT work. We trained this model using Reinforcement Learning from Human Feedback (RLHF), using the same methods as InstructGPT, but with slight differences in the data collection setup.