Current ChatGPT might be a great Maths teaching tool not because it never makes mistakes, but because it frequently does. Give it a problem, it generates convincing-looking but potentially bullsh*t answer, ask the student if they are convinced by the response.