The training of ChatGPT needed much less data than previous versions. They used reinforcement learning to train the model to give more "desirable" outputs according to human labelers that were probably somewhat leftwing.