The reason why ChatGPT lies is that there is no good reward function to support it. RLHF is one piece of the puzzle, we need more from knowledge base, interaction with the physical world, humans...Reinforcement Learning with All Feedba