You need a really good base model (aka GPT 3.5) in which to fine-tune/train to be good at dialogue. It started with GPT 3 InstructGPT. Those learnings informed the creation of ChatGPT. You need all those pieces and learnings to make RLHF impactful.