Latent Tree Models for Hierarchical Topic Detection
Table 1: Document partition given by latent variable Z21.
['[ITALIC] Z21', 's0 (0.95)', 's1 (0.05)']
There are totally 14 latent variables in the toy example. Each latent variable has two states and hence partitions the document collection into two soft clusters. To figure out what the partition and the two clusters are about, we need to consider the relationship between the latent variable and the observed variables in its subtree. Take Z21 as an example. Denote the two document clusters it gives as Z21= s0 and Z21=s1. We see that the cluster Z21=s1 consists of 5% of the documents. In this cluster, the words such as space, nasa and orbit occur with relatively high probabilities. It is clearly a meaningful and is interpreted as a topic. One might label the topic “NASA”. The other cluster Z21= s0 consists of 95% of the documents. In this cluster, the words occur with low probabilities. We interpret it as a background topic. There are three subtle issues concerning The first issue is how the word variables are ordered. To answer the question, we need the mutual information (MI) I(X;Y) The words placed at the top of the table have the highest MI with Z21. They are the best ones to characterize the difference between the two clusters because their occurrence probabilities in the two clusters differ the most. They occur with high probabilities in the clusters Z21=s1 and with low probabilities in Z21 =s0. If one is to choose only the top, say 5, words to characterize the topic Z21=s1, then the best words to pick are space, nasa, orbit, earth and shuttle. The computation takes linear time because the model is tree-structured.