Neural Machine Translation Training in a Multi-Domain Scenario
Table 4: Stacking versus concatenation
['[BOLD] Arabic-English', '[BOLD] Arabic-English ALL', '[BOLD] Arabic-English OD→TED', '[BOLD] Arabic-English UN→OPUS→TED']
We compared model stacking with different forms of concatenation. In terms of data usage, all models are exposed to identical data. The best systems are achieved using a concatenation of all of the out-of-domain data for initial model training and then fine-tuning the trained model on the in-domain data. The concatenated system ALL performed the lowest among all.