When ChatGPT was trained with RLHF, it's reward was based on predicting how the human would rate its response. Deceiving the humans would be part of the optimal strategy to maximize reward. That does not require