OpenAI havent released details of ChatGPT. We only know that its trained in the same way as InstructGPT. Our labelers prefer outputs from our 1.3B InstructGPT model over outputs from a 175B GPT-3 model, despite having