What You Say and How You Say it: Joint Modeling of Topics and Discourse in Microblog Conversations
Table 3: The purity, homogeneity, and variation of information (VI) scores for the latent discourse roles measured against the human-annotated dialogue acts. For purity and homogeneity, higher scores indicate better performance, while for VI scores, lower is better. In each column, the best results are in boldface. Our joint model Topic+Disc significantly outperforms all the baselines (p<0.01, paired t-test).
['[BOLD] Models', 'Purity', 'Homogeneity', 'VI']
For discourse evaluation, we measure whether the model-produced discourse assignments are consistent with the human-annotated dialogue acts. Hence following Zhao et al. Therefore, we adopt purity Manning et al. as our automatic evaluation metrics. Here, we set D=15 to ensure the number of latent discourse roles to be the same as the number of manually-labeled dialogue acts. Higher values indicate better performance for purity and homogeneity, while for VI, lower is better.