Learning Contextualized Sentence Representations for Document-Level Neural Machine Translation
Table 9: The number of parameters used in the baseline and our method and performance comparison.
['[BOLD] Methods', '[BOLD] Parameters', '[BOLD] BLEU']
The joint training method uses the same number of parameters as the baseline, achieving a higher BLEU score. The number of parameters in the pre-training & fine-tuning method is about 30% larger than that of the baseline due to the additional two encoders for the preceding and succeeding source sentence. If we do not pre-train the model, the performance significantly drops, even worse than the baseline. This suggests that the improvement of the pre-training & fine-tuning method over the baseline is achieved not because of using additional parameters but the learned contextual knowledge in the pre-training on the monolingual document data.