You may (should!) read this nice 74 year old paper. Among many other things expresses neatly that texts (i.e. sequences of symbols used to communicate ideas) are indeed random variables. As a side effect it helps understand better the LLMs s