ChatGPT sometimes writes plausible-sounding but incorrect or nonsensical answers. Fixing this issue is challenging, as: 1 during RL training theres currently no source of truth; 2 training the model to be more cautious causes it to decline questions that it can answer correctly;