Memory-Augmented Recurrent Neural Networks Can Learn Generalized Dyck Languages
Table 4: The performances of the vanilla and memory-augmented recurrent models on the D6. We note that the MARNNs in this example contain 12 hidden units and 5-dimensional external stack/memory. In 70 out of 100 trials, the MARNNs performed over 99% accuracy. Overall, the Baby-NTM+Softmax had the best performance.
['[BOLD] Models', '[BOLD] Training Set  [BOLD] Min', '[BOLD] Training Set  [BOLD] Max', '[BOLD] Training Set  [BOLD] Med', '[BOLD] Training Set  [BOLD] Mean', '[BOLD] Test Set  [BOLD] Min', '[BOLD] Test Set  [BOLD] Max', '[BOLD] Test Set  [BOLD] Med', '[BOLD] Test Set  [BOLD] Mean']
In contrast, the overall performance of the MARNNs for D6 was much lower than for D2 and D3. For instance, none of our models could obtain full accuracy on the training or test sets; the maximum score our models could achieve was 60.38%. We wondered whether increasing the dimension of the memory would remedy the problem. We saw a significant increase in the performance of our models: In 60 out of 90 trials, our enhanced MARNNs achieved almost perfect (≥99%) accuracy on the test set.