I've been trying out ChatGPT by asking some simple math questions. As one would expect of language-based models, it gets it wrong the vast majority of the time. There is a huge chasm between giving a linguistically coherent answer, and a correct one. A stochastic parrot indeed!