Roughly speaking (i.e., ignoring RLHF), ChatGPT produces a response that it thinks comes from the same distribution as the training corpus, but if the corpus is full of errors, the responses will be too. end/