The blog announcing ChatGPT talks about RLHF as the main thing they did on top of GPT-3.5. I think that was about taking the generic language model and making it give chat-like answers: helpful tone, the right length, etc.