I asked ChatGPT to calculate what resistors I needed for a voltage divider. It confidently and correctly explained Ohms law, then (equally confidently) gave me nonsense numbers that would have resulted in smoke.