This is called "hallucination" and I find it to be the biggest flaw with GPT-3 interfaces like ChatGPT. The problem is, these large language models are inherently kind of a black box, so how do you fix this kind of behavior?