Some of my first impressions when using ChatGPT have been confirmed by this nice paper that claims "ChatGPT's mathematical abilities are significantly below those of an average mathematics graduate student"