Neural Models for Documents with Metadata
Table 1: Performance of our various models in an unsupervised setting (i.e., without labels or covariates) on the IMDB dataset using a 5,000-word vocabulary and 50 topics. The supplementary materials contain additional results for 20 newsgroups and Yahoo answers.
['Model', 'Ppl. ↓', 'NPMI (int.) ↑', 'NPMI (ext.) ↑', 'Sparsity ↑']
Although the emphasis of this work is on incorporating observed labels and/or covariates, we briefly report on experiments in the unsupervised setting. Recall that, without metadata, scholar equates to ProdLDA, but with an explicit background term. We therefore use the same experimental setup as Srivastava and Sutton : our model returns more coherent topics than LDA, but at the cost of worse perplexity. SAGE, by contrast, attains very high levels of sparsity, but at the cost of worse perplexity and coherence than LDA. As expected, the NVDM produces relatively low perplexity, but very poor coherence, due to its lack of constraints on θ.