Dynamic Layer Aggregation for Neural Machine Translation with Routing-by-Agreement
Table 1: Translation performance on WMT14 English⇒German translation task. “# Para.” denotes the number of parameters, and “Train” and “Decode” respectively denote the training (steps/second) and decoding (sentences/second) speeds.
['[BOLD] #', '[BOLD] Model', '[BOLD] # Para.', '[BOLD] Train', '[BOLD] Decode', '[BOLD] BLEU', '△']
As one would expect, the linear combination (Row 2) improves translation performance by +0.42 BLEU points, indicating the necessity of aggregating layers for deep NMT models.