Neural Machine Translation Training in a Multi-Domain Scenario
Table 6: Comparing results of balanced ensemble (ENSb) and weighted ensemble (ENSw) with the best individual model and the concatenated model
['[BOLD] Arabic-English', '[BOLD] Arabic-English OPUS', '[BOLD] Arabic-English ALL', '[BOLD] Arabic-English ENS [ITALIC] b', '[BOLD] Arabic-English ENS [ITALIC] w']
We took the best model for every domain according to the average BLEU on the development sets and ensembled them during decoding. For weighted ensemble, we did a grid search and selected the weights using the development set. As expected, balanced ensemble (ENSb) dropped results compared to the best individual model. Since the domains are very distant, giving equal weights to them hurts the overall performance. The weighted ensemble (ENSw) improved from the best individual model by 1.8 BLEU points but is still lower than the concatenated system by 1.7 BLEU points. The weighted ensemble approach is beneficial when individual domain specific models are already available for testing. Decoding with multiple models is more efficient compared to training a system from scratch on a concatenation of the entire data.