I think ChatGPT (and other LLMs) is optimized for sounding convincingly human, not necessarily telling the truth. The RLHF training method selects for responses that sound good, but no fact-checking was done AFAIK.