LMs are trained on large datasets. OpenAI uses datasets from scans of the internet, in fact they used links sent through reddit comments with 2+ karma for GPT-2 excluding Wikipedia. Specifically ChatGPT trained