Understanding Neural Machine Translation by Simplification: The Case of Encoder-free Models
Table 3: BLEU scores of Trans-noEncs with different embedding initialization. “Random” means no pre-trained embeddings. “Fixed” and “Not-fixed” denote using pre-trained embeddings.
['Embeddings', 'Random', 'Fixed', 'Not-fixed']
In addition to the qualitative evaluation, we also conduct a quantitative evaluation. We first employ the learned embeddings from Transformer to initialize the embedding parameters in Trans-noEnc. The pre-trained embeddings can be either fixed or not fixed during training. The pre-trained embeddings slightly improve the BLEU score.