Spherical Latent Spaces for Stable Variational Autoencoders
Table 4: Test set perplexities for the document modeling task. Feedforward Deep Auto Regressive Neural Network (fDARN) is implemented by mnih2014neural. Gaussian-based NVDM (G-NVDM) is proposed in miao2016neural. Dim indicates the dimension of the latent code. Our v-NVDM model outperforms past models by a substantial margin.
['Model', 'Dim', '20NG', 'RCV1']
In contrast with NVRNN, the NVDM fully relies on the power of latent code to predict the word distribution, so we never observe a KL collapse, yet vMF still achieves better performance than Gaussian. For 20NG, which is much smaller than RCV1, smaller dimensions therefore give better performance. For both datasets, the settings of κ=100,dim=25 and κ=150,dim∈{50,200} work well.