I tried ChatGPT with the classic bat and ball problem. With an identical prompt each time it got the right answer half the time. All the other times it got wildly different results which might look plausible if you dont work out the problem yourself. Heres some wrong answers.