I tried the ChatGPT on the first Advent of Code puzzle input (with a prompt: "Write a python 3.9 program to answer this puzzle with the input in input.txt") and it produced close, uncanny valley answers that looked like they would work, but did not.