Recurrent Stacking of Layers for Compact Neural Machine Translation Models
Table 2: Results of decoding the GCP English-Japanese model for different levels of recurrence stacking than what was used for training.
['[BOLD] #recurrence for decoding', '[BOLD] #recurrently stacked layers in model  [BOLD] 1', '[BOLD] #recurrently stacked layers in model  [BOLD] 2', '[BOLD] #recurrently stacked layers in model  [BOLD] 3', '[BOLD] #recurrently stacked layers in model  [BOLD] 4', '[BOLD] #recurrently stacked layers in model  [BOLD] 5', '[BOLD] #recurrently stacked layers in model  [BOLD] 6']
In order to understand what each step of recurrent stacking brings about, we decided to train a N-layer recurrently stacked model and during decoding perform recurrence up to N-1 times. It can be seen that once the NMT model has been trained to use N layers of recurrent stacking, it is unable to perform optimally using fewer than N levels of stacking for decoding. Although, this is expected, there are three crucial observations as below.