Recurrent Graph Syntax Encoder for Neural Machine Translation
Table 1: Different settings that employed RGSE on different layer combinations in Transformer. “speed” denotes training speed measured in steps per second.
['[BOLD] #', '[BOLD] Layers', '[BOLD] Speed', '[BOLD] Val.', 'Δ']
\newcite anastasopoulos2018tied stated that high-level layers exploit more structure information and more long-distance dependencies than lower layers. We thus design ablation study to investigate if it is necessary to deploy RGSE on every layer. As is shown in Tab. This result is consistent with \newciteyang2018modeling ’s findings, validating our assumption.