Memory-Augmented Recurrent Neural Networks Can Learn Generalized Dyck Languages
Table 6: The performances of the vanilla and memory-augmented recurrent models on the string reversal task under the transduction setting. In 32 out of 90 trials, our MARNNs obtained perfect accuracy.
['[BOLD] Models', '[BOLD] Training Set  [BOLD] Min', '[BOLD] Training Set  [BOLD] Max', '[BOLD] Training Set  [BOLD] Med', '[BOLD] Training Set  [BOLD] Mean', '[BOLD] Test Set  [BOLD] Min', '[BOLD] Test Set  [BOLD] Max', '[BOLD] Test Set  [BOLD] Med', '[BOLD] Test Set  [BOLD] Mean']
The results corroborated our conjecture by showing that when the second half of the input samples contained symbols from an alphabet other than the one used in the first half (in this case, the # symbol), the memory-augmented models do not get confused and act in the desired way (pushing elements into the stack in the first half and popping them one by one in the second half after seeing the marker #). When we visualized the hidden states and the memory entries of the models for this task, we observed that our MARNNs learned to emulate simple pushdown-automata.