But if InstructGPT was trained on feedback, how is chatGPT so much better? There are a few changes. First, in the data collection setup - Supervised learning is used. Humans play both sides of the conversation. This is used to fine-tune the GPT 3.5 model.