Effective Estimation of Deep Generative Language Models
Table 5: Experimental settings.
['Parameter', 'Value']
To regularise the models, we use dropout with a shared mask across time-steps No dropout is applied to layers of the inference network as this does not lead to consistent empirical benefits and lacks a good theoretical basis. Gradient norms are clipped to prevent exploding gradients, and long sentences are truncated to three standard deviations above the average sentence length in the training data.