Compositional generalization in a deep seq2seq model by separating syntax and semantics
Table 5: Results of Syntactic Attention compared to models of Loula et al. [2018] on template splits of SCAN dataset. Mean accuracy (%) is shown with standard deviations. P = Primitive
['Model', 'Template split  [ITALIC] jump around right', 'Template split  [BOLD] P  [ITALIC] right', 'Template split  [BOLD] P  [ITALIC] opposite right', 'Template split  [BOLD] P  [ITALIC] around right']
The rest of the results are mixed: Syntactic Attention shows good compositional generalization performance on the Primitive right split, but fails on the Primitive opposite right and Primitive around right splits. All of the template tasks require models to generalize based on the symmetry between "left" and "right" in the dataset. However, in the opposite right and around right splits, this symmetry is substantially violated, as one of the two prepositional phrases in which they can occur is never seen with "right." Further research is required to determine whether a model implementing similar principles to Syntactic Attention can perform well on this task.