ChatGPT is essentially an alien mind. People can make a LLM using a transformer architecture and a bunch of other clever tricks, but no one actually understands what one will do until they've tried it. This is one of the main dangers of AI.