Echo State Neural Machine Translation
Table 1: BLEU score comparison on WMT testsets. LSTM is the fully-trainable baseline following RNMT+ settings, ESNMT(-LSTM) are proposed models with simple RNN or LSTM cells. Best ESNMT numbers are in bold.
['[BOLD] Model', '[BOLD] Dim', '[BOLD] EnFr', '[BOLD] EnDe', '[BOLD] EnRo']
To further investigate how spectral radius determines translation quality, we study BLEU scores on EnFr testset as a function of sentence length, using models in which radii ρl are fixed for all layers and set to 0.1, 0.9 or 2.0. The results are shown in Fig. in which remote inputs outweigh recent inputs, resulting in worse quality on short sentences. Radius 0.9 maintains a good balance between short and long memories, yielding the best quality.