There are significant additional costs to RLHF besides just having random ppl interact with your model ambiguously like in the ChatGPT setting Also there are many applications of LLMs that look nothing like ChatGPT and economics of data col