A Batch Normalized Inference Network Keeps the KL Vanishing Away
Table 3: Accuracy on Yelp.
['[BOLD] #label', '[BOLD] 100', '[BOLD] 500', '[BOLD] 1k', '[BOLD] 2k', '[BOLD] 10k']
Performance on a downstream task - Text classification: The goal of VAE is to learn a good representation of the data for downstream tasks. Here, we evaluate the quality of latent representations by training a one-layer linear classifier based on the mean of the posterior distribution. We use a downsampled version of the Yelp sentiment dataset shen2017style. li-etal-2019-surprisingly further sampled various labeled data to train the classifier. To compare with them fairly, we use the same samples in li-etal-2019-surprisingly. Our approach achieves the best accuracy in all the settings. For 10k training samples, all the methods get a good result. However, when only using 100 training samples, different methods vary a lot in accuracy. The text classification task shows that our approach can learn a good latent representation even without a pretrained inference network.