ChatGPT is trained by fine-tuning a large language model, GPT-3.5, using RLHF, which involves humans. The knowledge encoded in the underlying language model is the limiting factor regarding accuracy, width, and depth.