Zoneout: Regularizing RNNs by Randomly Preserving Hidden Activations
Table 2: Error rates on the pMNIST digit classification task. Zoneout outperforms recurrent dropout, and sets state of the art when combined with recurrent batch normalization.
['[BOLD] Model', '[BOLD] Valid', '[BOLD] Test']
However, by adding zoneout to the recurrent batch normalized LSTM, we achieve state of the art performance. For this setting, the zoneout mask is shared between cells and states, and the recurrent dropout probability and zoneout probabilities are both set to 0.15.