I was curious, so here's where ChatGPT gets its language data: English-language Wikipedia WebText2, a private OpenAI database of sites linked on Reddit with 3 upvotes. A public version: OpenWebtext2 Common Crawl, a bot that crawls the whole internet "Books1" "Books2"