Language models such as GPT are typically trained on large amounts of data that include books, articles and other written works, which allows them to better understand the patterns and structures of human language.