i believe it was trained on and uses Common Crawl? my basis for that is that's what GPT 3 and such were trained on, and I think ChatGPT is a version of GPT that was fine-tuned with RLHF. the problem is it's hard to know what *hasn't* been