For example, chatGPT generates fake code, but Github's co-pilot can generate lots of real code. Why is chatGPT prone to create nonsense? The same reason it is so remarkable at creative tasks: incredibly diverse training data.