CNNs found to jump around more skillfully than RNNs:Compositional generalization in seq2seq convolutional networks
Table 2: Test accuracy (%) on SCAN splits (means across 5 seeds, with standard deviation if available). Top LSTM results from Lake:Baroni:2017/Loula:etal:2018, GRU from Bastings:etal:2018.
['[EMPTY]', '[BOLD] random', '[BOLD] jump', '[BOLD] around-right']
CNNs, like RNNs, succeed in the random split, and achieve much higher accuracy (albeit still far from being perfect) in the challenging jump and around-right splits.