Neural networks model the probability distribution of their training data. Large language models like ChatGPT sample from these distributions. Hence can easily generate data that is not exactly in their training data.