The fundamental problem seems to be that a language model is a function from language language whereas people want to use it as if it were a model from knowledge language (e.g., when asking ChatGPT factual queries).