ChatGPT consistently returns the wrong answer to math problems, but its answer tends to remain within 10% of the correct answer. LLMs excel at estimation when it needs to learn to verify and correct false proofs.