Nonparametric Topic Modeling with Neural Inference
Table 1: Comparison of perplexity (lower is better) and topic coherence (higher is better) between different topic models on 20News and RCV1-V2 datasets.
['Methods', 'Perplexity 20News', 'Perplexity 20News', 'Perplexity RCV1-V2', 'Perplexity RCV1-V2', 'Coherence 20News', 'Coherence 20News', 'Coherence RCV1-V2', 'Coherence RCV1-V2']
We can clearly see that our models outperform the baselines, which indicates that our models have better goodness-to-fit capability and can discover topics that match more closely to human judgment. Note that comparing the ELBO-estimated perplexity of HiTM-VAE with other models directly is not suitable, as it has a lot more random variables, which usually leads to a higher ELBO. The possible reasons for the good coherence achieved by our models are 1) The “Product-of-Experts” enables the model to model sharper distributions. 2) The nonparametric characteristic means the models can adapt the number of topics to data, thus topics can be sufficiently trained and of high diversity. Table 1 in Appendix 1.1 illustrates the topics learned by iTM-VAE-Prod. Please refer to Appendix 1.1 in the supplementary for more details. In these baselines, at most 200 topics are used.