Information-theoretical analysis of the statistical dependencies among three variables: Applications to written language
Table 2: Pairs of words with highest mutual information. Left: OS. Right: AM. The values are in bits.
['[ITALIC] wi (OS) male', '[ITALIC] wj (OS) female', '[ITALIC] Iij 0.242', '[ITALIC] Hi 0.504', '[ITALIC] Hj 0.409', '[ITALIC] wi (AM) 1', '[ITALIC] wj (AM) 2', '[ITALIC] Iij 0.191', '[ITALIC] Hi 0.330', '[ITALIC] Hj 0.337']
In principle, there are two possible scenarios in which the mutual information between two variables can be high: (a) in each part of the book the two words either appear together or are both absent, and (b) the presence of one of the words in a given part excludes the presence of the other. In all these cases, the two words in the pair tend to be either simultaneously present or simultaneously absent (option (a) above). In both books, there are examples of words that participate in two pairs: cell is connected to both bee and wax (OS) and mnemic is connected to both phenomena and causation (AM). Their structure corresponds to the double links in the second and third columns of Figs. As explained in Sect. The words south and north are also linked to each other, but they only appear in position 32, with a mutual information that is approximately 1/3 of the two principal links. A similar situation is seen with bee and wax, both connected to cell, although the direct connection between bee and wax appears sooner, in position 16. The same happens in AM with phenomena and causation, linked through mnemic, which are connected to each other in the 39th place of the list. These examples pose the question whether the weakest link in the triangle could be entirely explained as a consequence of the two stronger links. A triplet analysis of pairwise interactions allows us to assess whether such is indeed the case (see Sect.