Model 3. Language models as a noisy repository of knowledge Language models are trained by trying to predict the next word in a text. It knows stuff like: The tallest mountain? ChatGPT: Mount Everest. Or "Olympus Mons"? It does not memorize 100% but has a broad understanding.