Deep Recurrent Models with Fast-Forward Connections for Neural Machine Translation
Table 3: BLEU scores with different LSTM layer width in Deep-Att. After using two times larger LSTM layer width of 1024, we can only obtain BLEU score of 33.8. It is still behind the corresponding Deep-Att with F-F.
['Models', 'F-F', '[ITALIC] ne', '[ITALIC] nd', 'width', 'BLEU']
Removing F-F connections also reduces the corresponding model size. In order to figure out the effect of F-F comparing models with the same parameter size, we increase the LSTM layer width of Deep-Att without F-F. In Tab. we show that, after using a two times larger LSTM layer width of 1024, we can only obtain a BLEU score of 33.8, which is still worse than the corresponding Deep-Att with F-F.