ChatGPT shows to me that an LLM is only as good as its training data. It clearly got trained on a lot of byte-sized code samples that are explained in-detail and is good at them. But it cannot "code". It can pattern match what it's already seen a lot of times.