I think the strongest argument that world modeling is possible with language alone is simply how capable models like ChatGPT are. Hard to imagine how it does some things if not with some kind of understanding of our world