Memory-Augmented Recurrent Neural Networks Can Learn Generalized Dyck Languages
Table 2: The performances of the vanilla and memory-augmented recurrent models on D2. Min/Max/Median/Mean results were obtained from 10 different runs of each model with the same random seed across each run. We note that both Stack-RNN+Softmax and Stack-RNN+Softmax-Temp achieved full accuracy on the test sets in 8 out of 10 times.
['[BOLD] Models', '[BOLD] Training Set  [BOLD] Min', '[BOLD] Training Set  [BOLD] Max', '[BOLD] Training Set  [BOLD] Med', '[BOLD] Training Set  [BOLD] Mean', '[BOLD] Test Set  [BOLD] Min', '[BOLD] Test Set  [BOLD] Max', '[BOLD] Test Set  [BOLD] Med', '[BOLD] Test Set  [BOLD] Mean']
Our empirical results highlight the dramatic performance difference between the memory-augmented recurrent networks and vanilla recurrent networks: We note that almost all our stack/memory-augmented architectures achieved full accuracy on the test set, which contained longer and deeper sequences than the training set, while the vanilla RNNs and LSTMs failed to generalize with below 5% accuracy. We further observe that the Stack-RNN proposed by Joulin and Mikolov