If you're playing with ChatGPT/GPT3.5, take a look at repo here: Pay close attention to the second to last example. You can't rely on these models to produce the truth even under simple circumstances.