Unsupervised Recurrent Neural Network Grammars
Table 5: Metrics related to the generative model/inference network for RNNG/URNNG. For the supervised RNNG we take the “inference network” to be the discriminative parser trained alongside the generative model (see section 3.3). Recon. PPL is the reconstruction perplexity based on Eqϕ(z|x)[logpθ(x|z)], and KL is the Kullback-Leibler divergence. Prior entropy is the entropy of the conditional prior pθ(z|x
['[EMPTY]', 'PTB RNNG', 'PTB URNNG', 'CTB RNNG', 'CTB URNNG']
The “reconstruction” perplexity based on Eqϕ(z|x)[logpθ(x|z)] is much lower than regular perplexity, and further, the Kullback-Leibler divergence between the conditional prior and the variational posterior, given by Eqϕ(z|x)[logqϕ(z|x)pθ(z|x