Perhaps the reason ChatGPT sometimes obfuscates its gaps in knowledge and logic is because it was trained via RL to produce the response that would get the highest human rating. This would bias it towards nice-sounding answers, rather than correct ones