The problem here is that tools like ChatGPT are confidently wrong. That is to say that these systems have no true concepts of what accurate looks like but have the ability to be very precise. This messes with our brains bc we frequently think precision is the same as accuracy