Natural-Parameter Networks: A Class of Probabilistic Neural Networks
Table 4: Link Rank on Three Datasets
['Method', 'SAE', 'SDAE', 'VAE', 'gamma NPN', 'Gaussian NPN', 'Poisson NPN']
For fair comparison we also try all baselines with double budget (a structure of B-200-50) and report whichever has higher accuracy. As we can see, by treating representations as distributions rather than points in a vector space, NPN is able to achieve much lower link rank than all baselines, including VAE with variance information. The numbers in the brackets show the link rank of NPN if we discard the variance information. The performance gain from variance information verifies the effectiveness of the variance (uncertainty) estimated by NPN. Among different variants of NPN, the Gaussian NPN seems to perform better in datasets with fewer words like Citeulike-t (only 18.8 words per document). The Poisson NPN, as a more natural choice to model text, achieves the best performance in datasets with more words (Citeulike-a and arXiv). As we can see, higher uncertainty often indicates not only higher reconstruction error E but also higher variance in E.