During the Reinforcement Learning from Human Feedback (RLHF) step, humans rewrote and rated the responses from the model. This feedback is used to improve the ChatGPT model iteratively, with the goal to produce answers which are highly rated by humans.