Today, tools like ChatGPT rely on a small number of trained human raters to judge the quality of a few thousand sample responses, a process known as reinforcement learning from human feedback (RLHF). And it works surprisingly well. But it doesnt learn from users in the wild.