Structured Attention Networks
Table 2: Performance (average length to failure %) of models on the tree-transduction task.
['Depth', 'No Atten', 'Simple', 'Structured']
Note that this task is fairly difficult as the encoder is quite simple. The baseline model (unsurprisingly) performs poorly as it has no information about the source ordering. The simple attention model performs better, but is significantly outperformed by the structured model with a tree structure bias. We hypothesize that the model is partially reconstructing the arithmetic tree.